{"thread":{"id":"47823","subject":"[PATCH 00/26] Moving global state into the repository object (part 1)","startedAt":"2018-02-13T01:22:53Z","lastAt":"2018-03-03T01:52:10Z","messageCount":239,"participants":["Stefan Beller","Duy Nguyen","Brandon Williams","Junio C Hamano","Jonathan Nieder","Jonathan Tan","Nguyễn Thái Ngọc Duy","brian m. carlson","Eric Sunshine","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":26},"messages":[{"id":"339134","messageId":"20180213012241.187007-1-sbeller@google.com","threadId":"47823","inReplyTo":null,"subject":"[PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:15Z","receivedAt":"2018-02-13T01:22:53Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"This is a real take on the first part of the recent RFC[1].\n\nJonathan Tan suggested[2] that \"sha1_loose_object_info to handle arbitrary repositories\"\nmight be a good breaking point for a first part at that RFC at patch 38.\nThis series is smaller and contains only 26 patches as the patches in the big\nRFC were slightly out of order.\n\nI developed this series partly by writing patches, but mostly by cherrypicking\nfrom that RFC on top of current master. I noticed no external conflicts apart\nfrom one addition to the repositories _INIT macro, which was easy to resolve.\n\nComments in the early range of that RFC were on 003 where Junio pointed out\nthat the coccinelle patch ought to be not in contrib/coccinelle, so I put it\nin a sub directory there, as 'make coccicheck' doesn't traverse subdirs.\n\nbrian had a questoin on patch 25 in the RFC, but that seemed to resolve itself\nwithout any suggestion to include into this series[3].\n\nDuy suggested that we shall not use the repository blindly, but should carefully\nexamine whether to pass on an object store or the refstore or such[4], which \nI agree with if it makes sense. This series unfortunately has an issue with that\nas I would not want to pass down the `ignore_env` flag separately from the object\nstore, so I made all functions that only take the object store to have the raw\nobject store as the first parameter, and others using the full repository.\n\nEric Sunshine brought up memory leaks with the RFC, and I would think to\nhave plugged all holes.\n\n[1] https://public-inbox.org/git/20180205235508.216277-1-sbeller@google.com/\n[2] https://public-inbox.org/git/20180207143300.ce1c39ca07f6a0d64fe0e7ca@google.com/\n[3] https://public-inbox.org/git/20180206011940.GD7904@genre.crustytoothpaste.net/\n[4] https://public-inbox.org/git/CACsJy8CGgekpX4cZkyyTSPrj87uQVKZSOL7fyT__P2dh_1LmVQ@mail.gmail.com/\n\nThanks,\nStefan\n\nJonathan Nieder (8):\n pack: move prepare_packed_git_run_once to object store\n pack: move approximate object count to object store\n sha1_file: add repository argument to sha1_file_name\n sha1_file: add repository argument to map_sha1_file\n sha1_file: allow stat_sha1_file to handle arbitrary repositories\n sha1_file: allow open_sha1_file to handle arbitrary repositories\n sha1_file: allow map_sha1_file_1 to handle arbitrary repositories\n sha1_file: allow sha1_loose_object_info to handle arbitrary\n repositories\n\nStefan Beller (18):\n repository: introduce raw object store field\n object-store: move alt_odb_list and alt_odb_tail to object store\n object-store: free alt_odb_list\n object-store: move packed_git and packed_git_mru to object store\n object-store: close all packs upon clearing the object store\n sha1_file: add raw_object_store argument to alt_odb_usable\n sha1_file: add repository argument to link_alt_odb_entry\n sha1_file: add repository argument to read_info_alternates\n sha1_file: add repository argument to link_alt_odb_entries\n sha1_file: add repository argument to prepare_alt_odb\n sha1_file: allow link_alt_odb_entries to handle arbitrary repositories\n sha1_file: allow prepare_alt_odb to handle arbitrary repositories\n sha1_file: add repository argument to stat_sha1_file\n sha1_file: add repository argument to open_sha1_file\n sha1_file: add repository argument to map_sha1_file_1\n sha1_file: add repository argument to sha1_loose_object_info\n sha1_file: allow sha1_file_name to handle arbitrary repositories\n sha1_file: allow map_sha1_file to handle arbitrary repositories\n\n builtin/am.c | 2 +-\n builtin/clone.c | 2 +-\n builtin/count-objects.c | 6 +-\n builtin/fetch.c | 2 +-\n builtin/fsck.c | 13 ++-\n builtin/gc.c | 4 +-\n builtin/grep.c | 2 +-\n builtin/index-pack.c | 1 +\n builtin/merge.c | 2 +-\n builtin/pack-objects.c | 21 ++--\n builtin/pack-redundant.c | 6 +-\n builtin/receive-pack.c | 3 +-\n cache.h | 46 ++------\n contrib/coccinelle/refactoring/packed_git.cocci | 7 ++\n environment.c | 5 +-\n fast-import.c | 6 +-\n http-backend.c | 6 +-\n http-push.c | 1 +\n http-walker.c | 4 +-\n http.c | 6 +-\n mru.h | 1 +\n object-store.h | 75 +++++++++++++\n object.c | 26 +++++\n pack-bitmap.c | 4 +-\n pack-check.c | 1 +\n pack-revindex.c | 1 +\n packfile.c | 64 +++++------\n packfile.h | 2 +-\n path.c | 2 +-\n reachable.c | 1 +\n repository.c | 17 ++-\n repository.h | 7 +-\n server-info.c | 6 +-\n sha1_file.c | 135 +++++++++++++-----------\n sha1_name.c | 11 +-\n streaming.c | 5 +-\n 36 files changed, 321 insertions(+), 182 deletions(-)\n create mode 100644 contrib/coccinelle/refactoring/packed_git.cocci\n create mode 100644 object-store.h\n\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339135","messageId":"20180213012241.187007-2-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 01/26] repository: introduce raw object store field","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:16Z","receivedAt":"2018-02-13T01:22:55Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The raw object store field will contain any objects needed for\naccess to objects in a given repository.\n\nThis patch introduces the raw object store and populates it with the\n`objectdir`, which used to be part of the repository struct.\n\nAs the struct gains members, we'll also populate the function to clear\nthe memory for these members.\n\nIn a later we'll introduce a struct object_parser, that will complement\nthe object parsing in a repository struct: The raw object parser is the\nlayer that will provide access to raw object content, while the higher\nlevel object parser code will parse raw objects and keeps track of\nparenthood and other object relationships using 'struct object'.\nFor now only add the lower level to the repository struct.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/grep.c | 2 +-\n environment.c | 5 +++--\n object-store.h | 15 +++++++++++++++\n object.c | 5 +++++\n path.c | 2 +-\n repository.c | 17 +++++++++++++----\n repository.h | 7 ++++---\n 7 files changed, 42 insertions(+), 11 deletions(-)\n create mode 100644 object-store.h\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 3ca4ac80d8..0f0c195705 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -432,7 +432,7 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject,\n \t * object.\n \t */\n \tgrep_read_lock();\n-\tadd_to_alternates_memory(submodule.objectdir);\n+\tadd_to_alternates_memory(submodule.objects.objectdir);\n \tgrep_read_unlock();\n \n \tif (oid) {\ndiff --git a/environment.c b/environment.c\nindex 63ac38a46f..5d7ffb3a3f 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -13,6 +13,7 @@\n #include \"refs.h\"\n #include \"fmt-merge-msg.h\"\n #include \"commit.h\"\n+#include \"object-store.h\"\n \n int trust_executable_bit = 1;\n int trust_ctime = 1;\n@@ -242,9 +243,9 @@ const char *get_git_work_tree(void)\n \n char *get_object_directory(void)\n {\n-\tif (!the_repository->objectdir)\n+\tif (!the_repository->objects.objectdir)\n \t\tBUG(\"git environment hasn't been setup\");\n-\treturn the_repository->objectdir;\n+\treturn the_repository->objects.objectdir;\n }\n \n int odb_mkstemp(struct strbuf *template, const char *pattern)\ndiff --git a/object-store.h b/object-store.h\nnew file mode 100644\nindex 0000000000..cf35760ceb\n--- /dev/null\n+++ b/object-store.h\n@@ -0,0 +1,15 @@\n+#ifndef OBJECT_STORE_H\n+#define OBJECT_STORE_H\n+\n+struct raw_object_store {\n+\t/*\n+\t * Path to the repository's object store.\n+\t * Cannot be NULL after initialization.\n+\t */\n+\tchar *objectdir;\n+};\n+#define RAW_OBJECT_STORE_INIT { NULL }\n+\n+void raw_object_store_clear(struct raw_object_store *o);\n+\n+#endif /* OBJECT_STORE_H */\ndiff --git a/object.c b/object.c\nindex 0afdfd19b7..9b5b65e189 100644\n--- a/object.c\n+++ b/object.c\n@@ -445,3 +445,8 @@ void clear_commit_marks_all(unsigned int flags)\n \t\t\tobj->flags &= ~flags;\n \t}\n }\n+\n+void raw_object_store_clear(struct raw_object_store *o)\n+{\n+\tfree(o->objectdir);\n+}\ndiff --git a/path.c b/path.c\nindex da8b655730..81a42d9115 100644\n--- a/path.c\n+++ b/path.c\n@@ -382,7 +382,7 @@ static void adjust_git_path(const struct repository *repo,\n \t\tstrbuf_splice(buf, 0, buf->len,\n \t\t\t repo->index_file, strlen(repo->index_file));\n \telse if (dir_prefix(base, \"objects\"))\n-\t\treplace_dir(buf, git_dir_len + 7, repo->objectdir);\n+\t\treplace_dir(buf, git_dir_len + 7, repo->objects.objectdir);\n \telse if (git_hooks_path && dir_prefix(base, \"hooks\"))\n \t\treplace_dir(buf, git_dir_len + 5, git_hooks_path);\n \telse if (repo->different_commondir)\ndiff --git a/repository.c b/repository.c\nindex f66fcb1342..bd2ad578de 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -1,11 +1,18 @@\n #include \"cache.h\"\n #include \"repository.h\"\n+#include \"object-store.h\"\n #include \"config.h\"\n #include \"submodule-config.h\"\n \n /* The main repository */\n static struct repository the_repo = {\n-\tNULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0\n+\tNULL, NULL,\n+\tRAW_OBJECT_STORE_INIT,\n+\tNULL, NULL, NULL,\n+\tNULL, NULL, NULL,\n+\t&the_index,\n+\t&hash_algos[GIT_HASH_SHA1],\n+\t0, 0\n };\n struct repository *the_repository = &the_repo;\n \n@@ -42,8 +49,8 @@ static void repo_setup_env(struct repository *repo)\n \t\t\t\t\t\t !repo->ignore_env);\n \tfree(repo->commondir);\n \trepo->commondir = strbuf_detach(&sb, NULL);\n-\tfree(repo->objectdir);\n-\trepo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,\n+\tfree(repo->objects.objectdir);\n+\trepo->objects.objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,\n \t\t\t\t\t \"objects\", !repo->ignore_env);\n \tfree(repo->graft_file);\n \trepo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,\n@@ -209,12 +216,14 @@ void repo_clear(struct repository *repo)\n {\n \tFREE_AND_NULL(repo->gitdir);\n \tFREE_AND_NULL(repo->commondir);\n-\tFREE_AND_NULL(repo->objectdir);\n \tFREE_AND_NULL(repo->graft_file);\n \tFREE_AND_NULL(repo->index_file);\n \tFREE_AND_NULL(repo->worktree);\n \tFREE_AND_NULL(repo->submodule_prefix);\n \n+\traw_object_store_clear(&repo->objects);\n+\tmemset(&repo->objects, 0, sizeof(repo->objects));\n+\n \tif (repo->config) {\n \t\tgit_configset_clear(repo->config);\n \t\tFREE_AND_NULL(repo->config);\ndiff --git a/repository.h b/repository.h\nindex 0329e40c7f..1f8bc7a7cf 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -1,6 +1,8 @@\n #ifndef REPOSITORY_H\n #define REPOSITORY_H\n \n+#include \"object-store.h\"\n+\n struct config_set;\n struct index_state;\n struct submodule_cache;\n@@ -21,10 +23,9 @@ struct repository {\n \tchar *commondir;\n \n \t/*\n-\t * Path to the repository's object store.\n-\t * Cannot be NULL after initialization.\n+\t * Holds any information related to the object store.\n \t */\n-\tchar *objectdir;\n+\tstruct raw_object_store objects;\n \n \t/*\n \t * Path to the repository's graft file.\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339136","messageId":"20180213012241.187007-3-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 02/26] object-store: move alt_odb_list and alt_odb_tail to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:17Z","receivedAt":"2018-02-13T01:22:57Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a process with multiple repositories open, alternates should be\nassociated to a single repository and not shared globally. Move\nalt_odb_list and alt_odb_tail into the_repository and adjust callers\nto reflect this.\n\nNow that the alternative object data base is per repository, we're\nleaking its memory upon freeing a repository. The next patch plugs\nthis hole.\n\nNo functional change intended.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/fsck.c | 4 +++-\n cache.h | 4 ++--\n object-store.h | 7 ++++++-\n packfile.c | 3 ++-\n sha1_file.c | 25 ++++++++++++-------------\n sha1_name.c | 3 ++-\n 6 files changed, 27 insertions(+), 19 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 04846d46f9..1048255da1 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -1,5 +1,6 @@\n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n #include \"commit.h\"\n #include \"tree.h\"\n@@ -694,7 +695,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t\tfsck_object_dir(get_object_directory());\n \n \t\tprepare_alt_odb();\n-\t\tfor (alt = alt_odb_list; alt; alt = alt->next)\n+\t\tfor (alt = the_repository->objects.alt_odb_list;\n+\t\t\t\talt; alt = alt->next)\n \t\t\tfsck_object_dir(alt->path);\n \n \t\tif (check_full) {\ndiff --git a/cache.h b/cache.h\nindex d8b975a571..918c2f15b4 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1573,7 +1573,7 @@ extern int has_dirs_only_path(const char *name, int len, int prefix_len);\n extern void schedule_dir_for_removal(const char *name, int len);\n extern void remove_scheduled_dirs(void);\n \n-extern struct alternate_object_database {\n+struct alternate_object_database {\n \tstruct alternate_object_database *next;\n \n \t/* see alt_scratch_buf() */\n@@ -1591,7 +1591,7 @@ extern struct alternate_object_database {\n \tstruct oid_array loose_objects_cache;\n \n \tchar path[FLEX_ARRAY];\n-} *alt_odb_list;\n+};\n extern void prepare_alt_odb(void);\n extern char *compute_alternate_path(const char *path, struct strbuf *err);\n typedef int alt_odb_fn(struct alternate_object_database *, void *);\ndiff --git a/object-store.h b/object-store.h\nindex cf35760ceb..44b8f84753 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -1,14 +1,19 @@\n #ifndef OBJECT_STORE_H\n #define OBJECT_STORE_H\n \n+#include \"cache.h\"\n+\n struct raw_object_store {\n \t/*\n \t * Path to the repository's object store.\n \t * Cannot be NULL after initialization.\n \t */\n \tchar *objectdir;\n+\n+\tstruct alternate_object_database *alt_odb_list;\n+\tstruct alternate_object_database **alt_odb_tail;\n };\n-#define RAW_OBJECT_STORE_INIT { NULL }\n+#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/packfile.c b/packfile.c\nindex 4a5fe7ab18..d61076faaf 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -1,6 +1,7 @@\n #include \"cache.h\"\n #include \"mru.h\"\n #include \"pack.h\"\n+#include \"repository.h\"\n #include \"dir.h\"\n #include \"mergesort.h\"\n #include \"packfile.h\"\n@@ -880,7 +881,7 @@ void prepare_packed_git(void)\n \t\treturn;\n \tprepare_packed_git_one(get_object_directory(), 1);\n \tprepare_alt_odb();\n-\tfor (alt = alt_odb_list; alt; alt = alt->next)\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)\n \t\tprepare_packed_git_one(alt->path, 0);\n \trearrange_packed_git();\n \tprepare_packed_git_mru();\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 3da70ac650..2826d5d6ed 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -22,6 +22,7 @@\n #include \"pack-revindex.h\"\n #include \"sha1-lookup.h\"\n #include \"bulk-checkin.h\"\n+#include \"repository.h\"\n #include \"streaming.h\"\n #include \"dir.h\"\n #include \"mru.h\"\n@@ -346,9 +347,6 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,\n \treturn buf->buf;\n }\n \n-struct alternate_object_database *alt_odb_list;\n-static struct alternate_object_database **alt_odb_tail;\n-\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\n@@ -368,7 +366,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)\n \t * Prevent the common mistake of listing the same\n \t * thing twice, or object directory itself.\n \t */\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\tif (!fspathcmp(path->buf, alt->path))\n \t\t\treturn 0;\n \t}\n@@ -428,8 +426,8 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,\n \tent = alloc_alt_odb(pathbuf.buf);\n \n \t/* add the alternate entry */\n-\t*alt_odb_tail = ent;\n-\talt_odb_tail = &(ent->next);\n+\t*the_repository->objects.alt_odb_tail = ent;\n+\tthe_repository->objects.alt_odb_tail = &(ent->next);\n \tent->next = NULL;\n \n \t/* recursively add alternates */\n@@ -563,7 +561,7 @@ void add_to_alternates_file(const char *reference)\n \t\tfprintf_or_die(out, \"%s\\n\", reference);\n \t\tif (commit_lock_file(&lock))\n \t\t\tdie_errno(\"unable to move new alternates file into place\");\n-\t\tif (alt_odb_tail)\n+\t\tif (the_repository->objects.alt_odb_tail)\n \t\t\tlink_alt_odb_entries(reference, '\\n', NULL, 0);\n \t}\n \tfree(alts);\n@@ -661,7 +659,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \tint r = 0;\n \n \tprepare_alt_odb();\n-\tfor (ent = alt_odb_list; ent; ent = ent->next) {\n+\tfor (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {\n \t\tr = fn(ent, cb);\n \t\tif (r)\n \t\t\tbreak;\n@@ -673,12 +671,13 @@ void prepare_alt_odb(void)\n {\n \tconst char *alt;\n \n-\tif (alt_odb_tail)\n+\tif (the_repository->objects.alt_odb_tail)\n \t\treturn;\n \n \talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n \n-\talt_odb_tail = &alt_odb_list;\n+\tthe_repository->objects.alt_odb_tail =\n+\t\t\t&the_repository->objects.alt_odb_list;\n \tlink_alt_odb_entries(alt, PATH_SEP, NULL, 0);\n \n \tread_info_alternates(get_object_directory(), 0);\n@@ -717,7 +716,7 @@ static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)\n {\n \tstruct alternate_object_database *alt;\n \tprepare_alt_odb();\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\tconst char *path = alt_sha1_path(alt, sha1);\n \t\tif (check_and_freshen_file(path, freshen))\n \t\t\treturn 1;\n@@ -873,7 +872,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n \n \tprepare_alt_odb();\n \terrno = ENOENT;\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tif (!lstat(*path, st))\n \t\t\treturn 0;\n@@ -899,7 +898,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \tmost_interesting_errno = errno;\n \n \tprepare_alt_odb();\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tfd = git_open(*path);\n \t\tif (fd >= 0)\ndiff --git a/sha1_name.c b/sha1_name.c\nindex 611c7d24dd..957ce25680 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -10,6 +10,7 @@\n #include \"dir.h\"\n #include \"sha1-array.h\"\n #include \"packfile.h\"\n+#include \"repository.h\"\n \n static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);\n \n@@ -104,7 +105,7 @@ static void find_short_object_filename(struct disambiguate_state *ds)\n \t\t */\n \t\tfakeent = alloc_alt_odb(get_object_directory());\n \t}\n-\tfakeent->next = alt_odb_list;\n+\tfakeent->next = the_repository->objects.alt_odb_list;\n \n \tfor (alt = fakeent; alt && !ds->ambiguous; alt = alt->next) {\n \t\tint pos;\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339137","messageId":"20180213012241.187007-4-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 03/26] object-store: free alt_odb_list","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:18Z","receivedAt":"2018-02-13T01:22:59Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Free the memory and reset alt_odb_{list, tail} to NULL.\n\nSigned-off-by: Stefan Beller \n---\n object.c | 17 +++++++++++++++++\n 1 file changed, 17 insertions(+)\n\ndiff --git a/object.c b/object.c\nindex 9b5b65e189..d778f09717 100644\n--- a/object.c\n+++ b/object.c\n@@ -446,7 +446,24 @@ void clear_commit_marks_all(unsigned int flags)\n \t}\n }\n \n+static void free_alt_odb(struct alternate_object_database *alt)\n+{\n+\tstrbuf_release(&alt->scratch);\n+\toid_array_clear(&alt->loose_objects_cache);\n+}\n+\n+static void free_alt_odbs(struct raw_object_store *o)\n+{\n+\twhile (o->alt_odb_list) {\n+\t\tfree_alt_odb(o->alt_odb_list);\n+\t\to->alt_odb_list = o->alt_odb_list->next;\n+\t}\n+}\n+\n void raw_object_store_clear(struct raw_object_store *o)\n {\n \tfree(o->objectdir);\n+\n+\tfree_alt_odbs(o);\n+\to->alt_odb_tail = NULL;\n }\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339138","messageId":"20180213012241.187007-6-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 05/26] object-store: close all packs upon clearing the object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:20Z","receivedAt":"2018-02-13T01:23:02Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n builtin/am.c | 2 +-\n builtin/clone.c | 2 +-\n builtin/fetch.c | 2 +-\n builtin/merge.c | 2 +-\n builtin/receive-pack.c | 2 +-\n object.c | 6 ++----\n packfile.c | 4 ++--\n packfile.h | 2 +-\n 8 files changed, 10 insertions(+), 12 deletions(-)\n\ndiff --git a/builtin/am.c b/builtin/am.c\nindex acfe9d3c8c..75e16dfec6 100644\n--- a/builtin/am.c\n+++ b/builtin/am.c\n@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)\n \t */\n \tif (!state->rebasing) {\n \t\tam_destroy(state);\n-\t\tclose_all_packs();\n+\t\tclose_all_packs(&the_repository->objects);\n \t\trun_command_v_opt(argv_gc_auto, RUN_GIT_CMD);\n \t}\n }\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 284651797e..7ec08d4696 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -1200,7 +1200,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \ttransport_disconnect(transport);\n \n \tif (option_dissociate) {\n-\t\tclose_all_packs();\n+\t\tclose_all_packs(&the_repository->objects);\n \t\tdissociate_from_references();\n \t}\n \ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 7bbcd26faf..70015ccdc7 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -1407,7 +1407,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n \n \tstring_list_clear(&list, 0);\n \n-\tclose_all_packs();\n+\tclose_all_packs(&the_repository->objects);\n \n \targv_array_pushl(&argv_gc_auto, \"gc\", \"--auto\", NULL);\n \tif (verbosity < 0)\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 30264cfd7c..907ae44ab5 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,\n \t\t\t * We ignore errors in 'gc --auto', since the\n \t\t\t * user should see them.\n \t\t\t */\n-\t\t\tclose_all_packs();\n+\t\t\tclose_all_packs(&the_repository->objects);\n \t\t\trun_command_v_opt(argv_gc_auto, RUN_GIT_CMD);\n \t\t}\n \t}\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex b2eac79a6e..954fc72c7c 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -2027,7 +2027,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)\n \t\t\tproc.git_cmd = 1;\n \t\t\tproc.argv = argv_gc_auto;\n \n-\t\t\tclose_all_packs();\n+\t\t\tclose_all_packs(&the_repository->objects);\n \t\t\tif (!start_command(&proc)) {\n \t\t\t\tif (use_sideband)\n \t\t\t\t\tcopy_to_sideband(proc.err, -1, NULL);\ndiff --git a/object.c b/object.c\nindex 81d083bec1..79c2c447bc 100644\n--- a/object.c\n+++ b/object.c\n@@ -4,6 +4,7 @@\n #include \"tree.h\"\n #include \"commit.h\"\n #include \"tag.h\"\n+#include \"packfile.h\"\n \n static struct object **obj_hash;\n static int nr_objs, obj_hash_size;\n@@ -468,8 +469,5 @@ void raw_object_store_clear(struct raw_object_store *o)\n \to->alt_odb_tail = NULL;\n \n \tmru_clear(&o->packed_git_mru);\n-\t/*\n-\t * TODO: call close_all_packs once migrated to\n-\t * take an object store argument\n-\t */\n+\tclose_all_packs(o);\n }\ndiff --git a/packfile.c b/packfile.c\nindex 11b0e6613c..2ce1b2cef7 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -306,11 +306,11 @@ static void close_pack(struct packed_git *p)\n \tclose_pack_index(p);\n }\n \n-void close_all_packs(void)\n+void close_all_packs(struct raw_object_store *o)\n {\n \tstruct packed_git *p;\n \n-\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n+\tfor (p = o->packed_git; p; p = p->next)\n \t\tif (p->do_not_close)\n \t\t\tdie(\"BUG: want to close pack marked 'do-not-close'\");\n \t\telse\ndiff --git a/packfile.h b/packfile.h\nindex 0cdeb54dcd..fa547226b7 100644\n--- a/packfile.h\n+++ b/packfile.h\n@@ -61,7 +61,7 @@ extern void close_pack_index(struct packed_git *);\n \n extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);\n extern void close_pack_windows(struct packed_git *);\n-extern void close_all_packs(void);\n+extern void close_all_packs(struct raw_object_store *o);\n extern void unuse_pack(struct pack_window **);\n extern void clear_delta_base_cache(void);\n extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339139","messageId":"20180213012241.187007-8-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 07/26] pack: move approximate object count to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:22Z","receivedAt":"2018-02-13T01:23:06Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Jonathan Nieder \n\nThe approximate_object_count() function maintains a rough count of\nobjects in a repository to estimate how long object name abbreviates\nshould be. Object names are scoped to a repository and the\nappropriate length may differ by repository, so the object count\nshould not be global.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 10 +++++++++-\n packfile.c | 11 +++++------\n 2 files changed, 14 insertions(+), 7 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 9fd109fa9c..86f1ec2cd0 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -21,13 +21,21 @@ struct raw_object_store {\n \tstruct alternate_object_database *alt_odb_list;\n \tstruct alternate_object_database **alt_odb_tail;\n \n+\t/*\n+\t * A fast, rough count of the number of objects in the repository.\n+\t * These two fields are not meant for direct access. Use\n+\t * approximate_object_count() instead.\n+\t */\n+\tunsigned long approximate_object_count;\n+\tunsigned approximate_object_count_valid : 1;\n+\n \t/*\n \t * Whether packed_git has already been populated with this repository's\n \t * packs.\n \t */\n \tunsigned packed_git_initialized : 1;\n };\n-#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_INIT, NULL, NULL, 0 }\n+#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_INIT, NULL, NULL, 0, 0, 0 }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/packfile.c b/packfile.c\nindex c107fce171..a53706cf16 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -793,8 +793,6 @@ static void prepare_packed_git_one(char *objdir, int local)\n \tstrbuf_release(&path);\n }\n \n-static int approximate_object_count_valid;\n-\n /*\n * Give a fast, rough count of the number of objects in the repository. This\n * ignores loose objects completely. If you have a lot of them, then either\n@@ -804,8 +802,8 @@ static int approximate_object_count_valid;\n */\n unsigned long approximate_object_count(void)\n {\n-\tstatic unsigned long count;\n-\tif (!approximate_object_count_valid) {\n+\tif (!the_repository->objects.approximate_object_count_valid) {\n+\t\tunsigned long count;\n \t\tstruct packed_git *p;\n \n \t\tprepare_packed_git();\n@@ -815,8 +813,9 @@ unsigned long approximate_object_count(void)\n \t\t\t\tcontinue;\n \t\t\tcount += p->num_objects;\n \t\t}\n+\t\tthe_repository->objects.approximate_object_count = count;\n \t}\n-\treturn count;\n+\treturn the_repository->objects.approximate_object_count;\n }\n \n static void *get_next_packed_git(const void *p)\n@@ -890,7 +889,7 @@ void prepare_packed_git(void)\n \n void reprepare_packed_git(void)\n {\n-\tapproximate_object_count_valid = 0;\n+\tthe_repository->objects.approximate_object_count_valid = 0;\n \tthe_repository->objects.packed_git_initialized = 0;\n \tprepare_packed_git();\n }\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339140","messageId":"20180213012241.187007-11-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 10/26] sha1_file: add repository argument to read_info_alternates","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:25Z","receivedAt":"2018-02-13T01:23:08Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"See previous patch for explanation.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 +++++++----\n 1 file changed, 7 insertions(+), 4 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 7dc5f690e2..4fdfdd945a 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -393,7 +393,9 @@ static int alt_odb_usable(struct raw_object_store *o,\n * SHA1, an extra slash for the first level indirection, and the\n * terminating NUL.\n */\n-static void read_info_alternates(const char * relative_base, int depth);\n+#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)\n+static void read_info_alternates_the_repository(const char *relative_base,\n+\t\t\t\t\t\tint depth);\n #define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)\n static int link_alt_odb_entry_the_repository(const char *entry,\n \tconst char *relative_base, int depth, const char *normalized_objdir)\n@@ -434,7 +436,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,\n \tent->next = NULL;\n \n \t/* recursively add alternates */\n-\tread_info_alternates(pathbuf.buf, depth + 1);\n+\tread_info_alternates(the_repository, pathbuf.buf, depth + 1);\n \n \tstrbuf_release(&pathbuf);\n \treturn 0;\n@@ -500,7 +502,8 @@ static void link_alt_odb_entries(const char *alt, int sep,\n \tstrbuf_release(&objdirbuf);\n }\n \n-static void read_info_alternates(const char * relative_base, int depth)\n+static void read_info_alternates_the_repository(const char *relative_base,\n+\t\t\t\t\t\tint depth)\n {\n \tchar *path;\n \tstruct strbuf buf = STRBUF_INIT;\n@@ -684,7 +687,7 @@ void prepare_alt_odb(void)\n \t\t\t&the_repository->objects.alt_odb_list;\n \tlink_alt_odb_entries(alt, PATH_SEP, NULL, 0);\n \n-\tread_info_alternates(get_object_directory(), 0);\n+\tread_info_alternates(the_repository, get_object_directory(), 0);\n }\n \n /* Returns 1 if we have successfully freshened the file, 0 otherwise. */\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339141","messageId":"20180213012241.187007-9-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 08/26] sha1_file: add raw_object_store argument to alt_odb_usable","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:23Z","receivedAt":"2018-02-13T01:23:10Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a raw_object_store to alt_odb_usable to be more specific about which\nrepository to act on. The choice of the repository is delegated to its\nonly caller link_alt_odb_entry.\n\nSigned-off-by: Stefan Beller \n---\n sha1_file.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 2826d5d6ed..d25b68a8fb 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -350,7 +350,9 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\n-static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)\n+static int alt_odb_usable(struct raw_object_store *o,\n+\t\t\t struct strbuf *path,\n+\t\t\t const char *normalized_objdir)\n {\n \tstruct alternate_object_database *alt;\n \n@@ -366,7 +368,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)\n \t * Prevent the common mistake of listing the same\n \t * thing twice, or object directory itself.\n \t */\n-\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = o->alt_odb_list; alt; alt = alt->next) {\n \t\tif (!fspathcmp(path->buf, alt->path))\n \t\t\treturn 0;\n \t}\n@@ -418,7 +420,7 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,\n \twhile (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')\n \t\tstrbuf_setlen(&pathbuf, pathbuf.len - 1);\n \n-\tif (!alt_odb_usable(&pathbuf, normalized_objdir)) {\n+\tif (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {\n \t\tstrbuf_release(&pathbuf);\n \t\treturn -1;\n \t}\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339142","messageId":"20180213012241.187007-12-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 11/26] sha1_file: add repository argument to link_alt_odb_entries","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:26Z","receivedAt":"2018-02-13T01:23:11Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"See previous patch for explanation.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 19 +++++++++++++------\n 1 file changed, 13 insertions(+), 6 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 4fdfdd945a..b090f403d8 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -471,8 +471,12 @@ static const char *parse_alt_odb_entry(const char *string,\n \treturn end;\n }\n \n-static void link_alt_odb_entries(const char *alt, int sep,\n-\t\t\t\t const char *relative_base, int depth)\n+#define link_alt_odb_entries(r, a, s, rb, d) \\\n+\tlink_alt_odb_entries_##r(a, s, rb, d)\n+static void link_alt_odb_entries_the_repository(const char *alt,\n+\t\t\t\t\t\tint sep,\n+\t\t\t\t\t\tconst char *relative_base,\n+\t\t\t\t\t\tint depth)\n {\n \tstruct strbuf objdirbuf = STRBUF_INIT;\n \tstruct strbuf entry = STRBUF_INIT;\n@@ -515,7 +519,7 @@ static void read_info_alternates_the_repository(const char *relative_base,\n \t\treturn;\n \t}\n \n-\tlink_alt_odb_entries(buf.buf, '\\n', relative_base, depth);\n+\tlink_alt_odb_entries(the_repository, buf.buf, '\\n', relative_base, depth);\n \tstrbuf_release(&buf);\n \tfree(path);\n }\n@@ -569,7 +573,8 @@ void add_to_alternates_file(const char *reference)\n \t\tif (commit_lock_file(&lock))\n \t\t\tdie_errno(\"unable to move new alternates file into place\");\n \t\tif (the_repository->objects.alt_odb_tail)\n-\t\t\tlink_alt_odb_entries(reference, '\\n', NULL, 0);\n+\t\t\tlink_alt_odb_entries(the_repository, reference,\n+\t\t\t\t\t '\\n', NULL, 0);\n \t}\n \tfree(alts);\n }\n@@ -582,7 +587,8 @@ void add_to_alternates_memory(const char *reference)\n \t */\n \tprepare_alt_odb();\n \n-\tlink_alt_odb_entries(reference, '\\n', NULL, 0);\n+\tlink_alt_odb_entries(the_repository, reference,\n+\t\t\t '\\n', NULL, 0);\n }\n \n /*\n@@ -685,7 +691,8 @@ void prepare_alt_odb(void)\n \n \tthe_repository->objects.alt_odb_tail =\n \t\t\t&the_repository->objects.alt_odb_list;\n-\tlink_alt_odb_entries(alt, PATH_SEP, NULL, 0);\n+\tlink_alt_odb_entries(the_repository, alt,\n+\t\t\t PATH_SEP, NULL, 0);\n \n \tread_info_alternates(the_repository, get_object_directory(), 0);\n }\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339143","messageId":"20180213012241.187007-10-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 09/26] sha1_file: add repository argument to link_alt_odb_entry","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:24Z","receivedAt":"2018-02-13T01:23:13Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the link_alt_odb_entry caller to be\nmore specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nSince the implementation does not yet work with other repositories,\nuse a wrapper macro to enforce that the caller passes in\nthe_repository as the first argument. It would be more appealing to\nuse BUILD_ASSERT_OR_ZERO to enforce this, but that doesn't work\nbecause it requires a compile-time constant and common compilers like\ngcc 4.8.4 do not consider \"r == the_repository\" a compile-time\nconstant.\n\nThis and the following three patches add repository arguments to\nlink_alt_odb_entry, read_info_alternates, link_alt_odb_entries\nand prepare_alt_odb. Three out of the four functions are found\nin a recursive call chain, calling each other, and one of them\naccesses the repositories `objectdir` (which was migrated; it\nwas an obvious choice) and `ignore_env` (which we need to keep in\nthe repository struct for clarify); hence we will pass through the\nrepository unlike just the object store object + the ignore_env flag.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d25b68a8fb..7dc5f690e2 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -394,8 +394,9 @@ static int alt_odb_usable(struct raw_object_store *o,\n * terminating NUL.\n */\n static void read_info_alternates(const char * relative_base, int depth);\n-static int link_alt_odb_entry(const char *entry, const char *relative_base,\n-\tint depth, const char *normalized_objdir)\n+#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)\n+static int link_alt_odb_entry_the_repository(const char *entry,\n+\tconst char *relative_base, int depth, const char *normalized_objdir)\n {\n \tstruct alternate_object_database *ent;\n \tstruct strbuf pathbuf = STRBUF_INIT;\n@@ -492,7 +493,8 @@ static void link_alt_odb_entries(const char *alt, int sep,\n \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n \t\tif (!entry.len)\n \t\t\tcontinue;\n-\t\tlink_alt_odb_entry(entry.buf, relative_base, depth, objdirbuf.buf);\n+\t\tlink_alt_odb_entry(the_repository, entry.buf,\n+\t\t\t\t relative_base, depth, objdirbuf.buf);\n \t}\n \tstrbuf_release(&entry);\n \tstrbuf_release(&objdirbuf);\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339144","messageId":"20180213012241.187007-16-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 15/26] sha1_file: add repository argument to sha1_file_name","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:30Z","receivedAt":"2018-02-13T01:23:16Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Jonathan Nieder \n\nAdd a repository argument to allow sha1_file_name callers to be more\nspecific about which repository to handle. This is a small mechanical\nchange; it doesn't change the implementation to handle repositories\nother than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nWhile at it, move the declaration to object-store.h, where it should\nbe easier to find.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n cache.h | 8 --------\n http-walker.c | 3 ++-\n http.c | 5 +++--\n object-store.h | 9 +++++++++\n sha1_file.c | 11 ++++++-----\n 5 files changed, 20 insertions(+), 16 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 09d37cb651..4b01c0de10 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -956,14 +956,6 @@ extern void check_repository_format(void);\n #define DATA_CHANGED 0x0020\n #define TYPE_CHANGED 0x0040\n \n-/*\n- * Return the name of the file in the local object database that would\n- * be used to store a loose object with the specified sha1. The\n- * return value is a pointer to a statically allocated buffer that is\n- * overwritten each time the function is called.\n- */\n-extern const char *sha1_file_name(const unsigned char *sha1);\n-\n /*\n * Return an abbreviated sha1 unique within this repository's object database.\n * The result will be at least `len` characters long, and will be NUL\ndiff --git a/http-walker.c b/http-walker.c\nindex a1c6f2639b..96873bdfed 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -1,4 +1,5 @@\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"commit.h\"\n #include \"walker.h\"\n #include \"http.h\"\n@@ -546,7 +547,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1)\n \t\tret = error(\"File %s has bad hash\", hex);\n \t} else if (req->rename < 0) {\n \t\tret = error(\"unable to write sha1 filename %s\",\n-\t\t\t sha1_file_name(req->sha1));\n+\t\t\t sha1_file_name(the_repository, req->sha1));\n \t}\n \n \trelease_http_object_request(req);\ndiff --git a/http.c b/http.c\nindex ab989b88dd..c7d2b74ce2 100644\n--- a/http.c\n+++ b/http.c\n@@ -2181,7 +2181,7 @@ struct http_object_request *new_http_object_request(const char *base_url,\n \thashcpy(freq->sha1, sha1);\n \tfreq->localfile = -1;\n \n-\tfilename = sha1_file_name(sha1);\n+\tfilename = sha1_file_name(the_repository, sha1);\n \tsnprintf(freq->tmpfile, sizeof(freq->tmpfile),\n \t\t \"%s.temp\", filename);\n \n@@ -2329,7 +2329,8 @@ int finish_http_object_request(struct http_object_request *freq)\n \t\treturn -1;\n \t}\n \tfreq->rename =\n-\t\tfinalize_object_file(freq->tmpfile, sha1_file_name(freq->sha1));\n+\t\tfinalize_object_file(freq->tmpfile,\n+\t\t\t\t sha1_file_name(the_repository, freq->sha1));\n \n \treturn freq->rename;\n }\ndiff --git a/object-store.h b/object-store.h\nindex add1d4e27c..5422e80c08 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -61,6 +61,15 @@ struct packed_git {\n \tchar pack_name[FLEX_ARRAY]; /* more */\n };\n \n+/*\n+ * Return the name of the file in a repository's local object database\n+ * that would be used to store a loose object with the specified sha1.\n+ * The return value is a pointer to a statically allocated buffer that\n+ * is overwritten each time the function is called.\n+ */\n+#define sha1_file_name(r, s) sha1_file_name_##r(s)\n+const char *sha1_file_name_the_repository(const unsigned char *sha1);\n+\n void prepare_alt_odb(struct repository *r);\n \n #endif /* OBJECT_STORE_H */\ndiff --git a/sha1_file.c b/sha1_file.c\nindex f046d560f8..65fefdf4ac 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -323,7 +323,7 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)\n \t}\n }\n \n-const char *sha1_file_name(const unsigned char *sha1)\n+const char *sha1_file_name_the_repository(const unsigned char *sha1)\n {\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n@@ -721,7 +721,8 @@ int check_and_freshen_file(const char *fn, int freshen)\n \n static int check_and_freshen_local(const unsigned char *sha1, int freshen)\n {\n-\treturn check_and_freshen_file(sha1_file_name(sha1), freshen);\n+\treturn check_and_freshen_file(sha1_file_name(the_repository, sha1),\n+\t\t\t\t freshen);\n }\n \n static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)\n@@ -878,7 +879,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n {\n \tstruct alternate_object_database *alt;\n \n-\t*path = sha1_file_name(sha1);\n+\t*path = sha1_file_name(the_repository, sha1);\n \tif (!lstat(*path, st))\n \t\treturn 0;\n \n@@ -903,7 +904,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \tstruct alternate_object_database *alt;\n \tint most_interesting_errno;\n \n-\t*path = sha1_file_name(sha1);\n+\t*path = sha1_file_name(the_repository, sha1);\n \tfd = git_open(*path);\n \tif (fd >= 0)\n \t\treturn fd;\n@@ -1568,7 +1569,7 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,\n \tgit_SHA_CTX c;\n \tunsigned char parano_sha1[20];\n \tstatic struct strbuf tmp_file = STRBUF_INIT;\n-\tconst char *filename = sha1_file_name(sha1);\n+\tconst char *filename = sha1_file_name(the_repository, sha1);\n \n \tfd = create_tmpfile(&tmp_file, filename);\n \tif (fd < 0) {\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339145","messageId":"20180213012241.187007-15-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 14/26] sha1_file: allow prepare_alt_odb to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:29Z","receivedAt":"2018-02-13T01:23:22Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n object-store.h | 3 +--\n sha1_file.c | 21 +++++++++++----------\n 2 files changed, 12 insertions(+), 12 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex d96a16edd1..add1d4e27c 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -61,7 +61,6 @@ struct packed_git {\n \tchar pack_name[FLEX_ARRAY]; /* more */\n };\n \n-#define prepare_alt_odb(r) prepare_alt_odb_##r()\n-extern void prepare_alt_odb_the_repository(void);\n+void prepare_alt_odb(struct repository *r);\n \n #endif /* OBJECT_STORE_H */\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d18ce3aeba..f046d560f8 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -677,21 +677,22 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \treturn r;\n }\n \n-void prepare_alt_odb_the_repository(void)\n+void prepare_alt_odb(struct repository *r)\n {\n-\tconst char *alt;\n-\n-\tif (the_repository->objects.alt_odb_tail)\n+\tif (r->objects.alt_odb_tail)\n \t\treturn;\n \n-\talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n+\tr->objects.alt_odb_tail = &r->objects.alt_odb_list;\n+\n+\tif (!r->ignore_env) {\n+\t\tconst char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);\n+\t\tif (!alt)\n+\t\t\talt = \"\";\n \n-\tthe_repository->objects.alt_odb_tail =\n-\t\t\t&the_repository->objects.alt_odb_list;\n-\tlink_alt_odb_entries(the_repository, alt,\n-\t\t\t PATH_SEP, NULL, 0);\n+\t\tlink_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);\n+\t}\n \n-\tread_info_alternates(the_repository, get_object_directory(), 0);\n+\tread_info_alternates(r, r->objects.objectdir, 0);\n }\n \n /* Returns 1 if we have successfully freshened the file, 0 otherwise. */\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339146","messageId":"20180213012241.187007-20-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 19/26] sha1_file: add repository argument to map_sha1_file","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:34Z","receivedAt":"2018-02-13T01:23:23Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Jonathan Nieder \n\nAdd a repository argument to allow map_sha1_file callers to be more\nspecific about which repository to handle. This is a small mechanical\nchange; it doesn't change the implementation to handle repositories\nother than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nWhile at it, move the declaration to object-store.h, where it should\nbe easier to find.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n cache.h | 1 -\n object-store.h | 2 ++\n sha1_file.c | 4 ++--\n streaming.c | 5 ++++-\n 4 files changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 4b01c0de10..d0a1236dac 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1237,7 +1237,6 @@ extern int pretend_sha1_file(void *, unsigned long, enum object_type, unsigned c\n extern int force_object_loose(const unsigned char *sha1, time_t mtime);\n extern int git_open_cloexec(const char *name, int flags);\n #define git_open(name) git_open_cloexec(name, O_RDONLY)\n-extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size);\n extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);\n extern int parse_sha1_header(const char *hdr, unsigned long *sizep);\n \ndiff --git a/object-store.h b/object-store.h\nindex 5422e80c08..0a4561f476 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -69,6 +69,8 @@ struct packed_git {\n */\n #define sha1_file_name(r, s) sha1_file_name_##r(s)\n const char *sha1_file_name_the_repository(const unsigned char *sha1);\n+#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)\n+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);\n \n void prepare_alt_odb(struct repository *r);\n \ndiff --git a/sha1_file.c b/sha1_file.c\nindex f706526bc2..1ce4058644 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -960,7 +960,7 @@ static void *map_sha1_file_1_the_repository(const char *path,\n \treturn map;\n }\n \n-void *map_sha1_file(const unsigned char *sha1, unsigned long *size)\n+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)\n {\n \treturn map_sha1_file_1(the_repository, NULL, sha1, size);\n }\n@@ -1184,7 +1184,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,\n \t\treturn 0;\n \t}\n \n-\tmap = map_sha1_file(sha1, &mapsize);\n+\tmap = map_sha1_file(the_repository, sha1, &mapsize);\n \tif (!map)\n \t\treturn -1;\n \ndiff --git a/streaming.c b/streaming.c\nindex 5892b50bd8..22d27df55e 100644\n--- a/streaming.c\n+++ b/streaming.c\n@@ -3,6 +3,8 @@\n */\n #include \"cache.h\"\n #include \"streaming.h\"\n+#include \"repository.h\"\n+#include \"object-store.h\"\n #include \"packfile.h\"\n \n enum input_source {\n@@ -335,7 +337,8 @@ static struct stream_vtbl loose_vtbl = {\n \n static open_method_decl(loose)\n {\n-\tst->u.loose.mapped = map_sha1_file(sha1, &st->u.loose.mapsize);\n+\tst->u.loose.mapped = map_sha1_file(the_repository,\n+\t\t\t\t\t sha1, &st->u.loose.mapsize);\n \tif (!st->u.loose.mapped)\n \t\treturn -1;\n \tif ((unpack_sha1_header(&st->z,\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339147","messageId":"20180213012241.187007-21-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 20/26] sha1_file: add repository argument to sha1_loose_object_info","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:35Z","receivedAt":"2018-02-13T01:23:28Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the sha1_loose_object_info caller\nto be more specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 1ce4058644..d0bc09a089 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -1151,9 +1151,10 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)\n \treturn parse_sha1_header_extended(hdr, &oi, 0);\n }\n \n-static int sha1_loose_object_info(const unsigned char *sha1,\n-\t\t\t\t struct object_info *oi,\n-\t\t\t\t int flags)\n+#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)\n+static int sha1_loose_object_info_the_repository(const unsigned char *sha1,\n+\t\t\t\t\t\t struct object_info *oi,\n+\t\t\t\t\t\t int flags)\n {\n \tint status = 0;\n \tunsigned long mapsize;\n@@ -1266,7 +1267,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,\n \n \tif (!find_pack_entry(real, &e)) {\n \t\t/* Most likely it's a loose object. */\n-\t\tif (!sha1_loose_object_info(real, oi, flags))\n+\t\tif (!sha1_loose_object_info(the_repository, real, oi, flags))\n \t\t\treturn 0;\n \n \t\t/* Not a loose object; someone else may have just packed it. */\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339148","messageId":"20180213012241.187007-22-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 21/26] sha1_file: allow sha1_file_name to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:36Z","receivedAt":"2018-02-13T01:23:30Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 3 +--\n sha1_file.c | 4 ++--\n 2 files changed, 3 insertions(+), 4 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 0a4561f476..b4756444bb 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -67,8 +67,7 @@ struct packed_git {\n * The return value is a pointer to a statically allocated buffer that\n * is overwritten each time the function is called.\n */\n-#define sha1_file_name(r, s) sha1_file_name_##r(s)\n-const char *sha1_file_name_the_repository(const unsigned char *sha1);\n+const char *sha1_file_name(struct repository *r, const unsigned char *sha1);\n #define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)\n void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);\n \ndiff --git a/sha1_file.c b/sha1_file.c\nindex d0bc09a089..fddada5756 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -323,12 +323,12 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)\n \t}\n }\n \n-const char *sha1_file_name_the_repository(const unsigned char *sha1)\n+const char *sha1_file_name(struct repository *r, const unsigned char *sha1)\n {\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tstrbuf_addf(&buf, \"%s/\", get_object_directory());\n+\tstrbuf_addf(&buf, \"%s/\", r->objects.objectdir);\n \n \tfill_sha1_path(&buf, sha1);\n \treturn buf.buf;\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339149","messageId":"20180213012241.187007-23-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 22/26] sha1_file: allow stat_sha1_file to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:37Z","receivedAt":"2018-02-13T01:23:31Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Jonathan Nieder \n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 +++++------\n 1 file changed, 5 insertions(+), 6 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex fddada5756..21ddbff846 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -874,19 +874,18 @@ int git_open_cloexec(const char *name, int flags)\n * Note that it may point to static storage and is only valid until another\n * call to sha1_file_name(), etc.\n */\n-#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)\n-static int stat_sha1_file_the_repository(const unsigned char *sha1,\n-\t\t\t\t\t struct stat *st, const char **path)\n+static int stat_sha1_file(struct repository *r, const unsigned char *sha1,\n+\t\t\t struct stat *st, const char **path)\n {\n \tstruct alternate_object_database *alt;\n \n-\t*path = sha1_file_name(the_repository, sha1);\n+\t*path = sha1_file_name(r, sha1);\n \tif (!lstat(*path, st))\n \t\treturn 0;\n \n-\tprepare_alt_odb(the_repository);\n+\tprepare_alt_odb(r);\n \terrno = ENOENT;\n-\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = r->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tif (!lstat(*path, st))\n \t\t\treturn 0;\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339150","messageId":"20180213012241.187007-25-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 24/26] sha1_file: allow map_sha1_file_1 to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:39Z","receivedAt":"2018-02-13T01:23:34Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Jonathan Nieder \n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 8 +++-----\n 1 file changed, 3 insertions(+), 5 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 50202f0959..86b0ca7089 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -928,10 +928,8 @@ static int open_sha1_file(struct repository *r,\n * Map the loose object at \"path\" if it is not NULL, or the path found by\n * searching for a loose object named \"sha1\".\n */\n-#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)\n-static void *map_sha1_file_1_the_repository(const char *path,\n-\t\t\t\t\t const unsigned char *sha1,\n-\t\t\t\t\t unsigned long *size)\n+static void *map_sha1_file_1(struct repository *r, const char *path,\n+\t\t\t const unsigned char *sha1, unsigned long *size)\n {\n \tvoid *map;\n \tint fd;\n@@ -939,7 +937,7 @@ static void *map_sha1_file_1_the_repository(const char *path,\n \tif (path)\n \t\tfd = git_open(path);\n \telse\n-\t\tfd = open_sha1_file(the_repository, sha1, &path);\n+\t\tfd = open_sha1_file(r, sha1, &path);\n \tmap = NULL;\n \tif (fd >= 0) {\n \t\tstruct stat st;\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339151","messageId":"20180213012241.187007-27-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 26/26] sha1_file: allow sha1_loose_object_info to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:41Z","receivedAt":"2018-02-13T01:23:38Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Jonathan Nieder \n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 +++++------\n 1 file changed, 5 insertions(+), 6 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 6237d59a59..c3f35914ce 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -1148,10 +1148,9 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)\n \treturn parse_sha1_header_extended(hdr, &oi, 0);\n }\n \n-#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)\n-static int sha1_loose_object_info_the_repository(const unsigned char *sha1,\n-\t\t\t\t\t\t struct object_info *oi,\n-\t\t\t\t\t\t int flags)\n+static int sha1_loose_object_info(struct repository *r,\n+\t\t\t\t const unsigned char *sha1,\n+\t\t\t\t struct object_info *oi, int flags)\n {\n \tint status = 0;\n \tunsigned long mapsize;\n@@ -1175,14 +1174,14 @@ static int sha1_loose_object_info_the_repository(const unsigned char *sha1,\n \tif (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {\n \t\tconst char *path;\n \t\tstruct stat st;\n-\t\tif (stat_sha1_file(the_repository, sha1, &st, &path) < 0)\n+\t\tif (stat_sha1_file(r, sha1, &st, &path) < 0)\n \t\t\treturn -1;\n \t\tif (oi->disk_sizep)\n \t\t\t*oi->disk_sizep = st.st_size;\n \t\treturn 0;\n \t}\n \n-\tmap = map_sha1_file(the_repository, sha1, &mapsize);\n+\tmap = map_sha1_file(r, sha1, &mapsize);\n \tif (!map)\n \t\treturn -1;\n \n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339152","messageId":"20180213012241.187007-24-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 23/26] sha1_file: allow open_sha1_file to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:38Z","receivedAt":"2018-02-13T01:23:40Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Jonathan Nieder \n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 +++++------\n 1 file changed, 5 insertions(+), 6 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 21ddbff846..50202f0959 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -898,22 +898,21 @@ static int stat_sha1_file(struct repository *r, const unsigned char *sha1,\n * Like stat_sha1_file(), but actually open the object and return the\n * descriptor. See the caveats on the \"path\" parameter above.\n */\n-#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)\n-static int open_sha1_file_the_repository(const unsigned char *sha1,\n-\t\t\t\t\t const char **path)\n+static int open_sha1_file(struct repository *r,\n+\t\t\t const unsigned char *sha1, const char **path)\n {\n \tint fd;\n \tstruct alternate_object_database *alt;\n \tint most_interesting_errno;\n \n-\t*path = sha1_file_name(the_repository, sha1);\n+\t*path = sha1_file_name(r, sha1);\n \tfd = git_open(*path);\n \tif (fd >= 0)\n \t\treturn fd;\n \tmost_interesting_errno = errno;\n \n-\tprepare_alt_odb(the_repository);\n-\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n+\tprepare_alt_odb(r);\n+\tfor (alt = r->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tfd = git_open(*path);\n \t\tif (fd >= 0)\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339153","messageId":"20180213012241.187007-26-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 25/26] sha1_file: allow map_sha1_file to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:40Z","receivedAt":"2018-02-13T01:23:42Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 3 +--\n sha1_file.c | 5 +++--\n 2 files changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex b4756444bb..f164c4e5c9 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -68,8 +68,7 @@ struct packed_git {\n * is overwritten each time the function is called.\n */\n const char *sha1_file_name(struct repository *r, const unsigned char *sha1);\n-#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)\n-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);\n+extern void *map_sha1_file(struct repository *r, const unsigned char *sha1, unsigned long *size);\n \n void prepare_alt_odb(struct repository *r);\n \ndiff --git a/sha1_file.c b/sha1_file.c\nindex 86b0ca7089..6237d59a59 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -956,9 +956,10 @@ static void *map_sha1_file_1(struct repository *r, const char *path,\n \treturn map;\n }\n \n-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)\n+void *map_sha1_file(struct repository *r,\n+\t\t const unsigned char *sha1, unsigned long *size)\n {\n-\treturn map_sha1_file_1(the_repository, NULL, sha1, size);\n+\treturn map_sha1_file_1(r, NULL, sha1, size);\n }\n \n static int unpack_sha1_short_header(git_zstream *stream,\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339154","messageId":"20180213012241.187007-17-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 16/26] sha1_file: add repository argument to stat_sha1_file","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:31Z","receivedAt":"2018-02-13T01:23:46Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the stat_sha1_file caller to be\nmore specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 65fefdf4ac..1d81bafe56 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -874,8 +874,9 @@ int git_open_cloexec(const char *name, int flags)\n * Note that it may point to static storage and is only valid until another\n * call to sha1_file_name(), etc.\n */\n-static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n-\t\t\t const char **path)\n+#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)\n+static int stat_sha1_file_the_repository(const unsigned char *sha1,\n+\t\t\t\t\t struct stat *st, const char **path)\n {\n \tstruct alternate_object_database *alt;\n \n@@ -1173,7 +1174,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,\n \tif (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {\n \t\tconst char *path;\n \t\tstruct stat st;\n-\t\tif (stat_sha1_file(sha1, &st, &path) < 0)\n+\t\tif (stat_sha1_file(the_repository, sha1, &st, &path) < 0)\n \t\t\treturn -1;\n \t\tif (oi->disk_sizep)\n \t\t\t*oi->disk_sizep = st.st_size;\n@@ -1372,7 +1373,7 @@ void *read_sha1_file_extended(const unsigned char *sha1,\n \t\tdie(\"replacement %s not found for %s\",\n \t\t sha1_to_hex(repl), sha1_to_hex(sha1));\n \n-\tif (!stat_sha1_file(repl, &st, &path))\n+\tif (!stat_sha1_file(the_repository, repl, &st, &path))\n \t\tdie(\"loose object %s (stored in %s) is corrupt\",\n \t\t sha1_to_hex(repl), path);\n \n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339155","messageId":"20180213012241.187007-18-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 17/26] sha1_file: add repository argument to open_sha1_file","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:32Z","receivedAt":"2018-02-13T01:23:48Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the open_sha1_file caller to be\nmore specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 1d81bafe56..e9e01fb471 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -899,7 +899,9 @@ static int stat_sha1_file_the_repository(const unsigned char *sha1,\n * Like stat_sha1_file(), but actually open the object and return the\n * descriptor. See the caveats on the \"path\" parameter above.\n */\n-static int open_sha1_file(const unsigned char *sha1, const char **path)\n+#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)\n+static int open_sha1_file_the_repository(const unsigned char *sha1,\n+\t\t\t\t\t const char **path)\n {\n \tint fd;\n \tstruct alternate_object_database *alt;\n@@ -938,7 +940,7 @@ static void *map_sha1_file_1(const char *path,\n \tif (path)\n \t\tfd = git_open(path);\n \telse\n-\t\tfd = open_sha1_file(sha1, &path);\n+\t\tfd = open_sha1_file(the_repository, sha1, &path);\n \tmap = NULL;\n \tif (fd >= 0) {\n \t\tstruct stat st;\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339156","messageId":"20180213012241.187007-19-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 18/26] sha1_file: add repository argument to map_sha1_file_1","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:33Z","receivedAt":"2018-02-13T01:23:49Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the map_sha1_file_1 caller to be\nmore specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 ++++++-----\n 1 file changed, 6 insertions(+), 5 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex e9e01fb471..f706526bc2 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -930,9 +930,10 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,\n * Map the loose object at \"path\" if it is not NULL, or the path found by\n * searching for a loose object named \"sha1\".\n */\n-static void *map_sha1_file_1(const char *path,\n-\t\t\t const unsigned char *sha1,\n-\t\t\t unsigned long *size)\n+#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)\n+static void *map_sha1_file_1_the_repository(const char *path,\n+\t\t\t\t\t const unsigned char *sha1,\n+\t\t\t\t\t unsigned long *size)\n {\n \tvoid *map;\n \tint fd;\n@@ -961,7 +962,7 @@ static void *map_sha1_file_1(const char *path,\n \n void *map_sha1_file(const unsigned char *sha1, unsigned long *size)\n {\n-\treturn map_sha1_file_1(NULL, sha1, size);\n+\treturn map_sha1_file_1(the_repository, NULL, sha1, size);\n }\n \n static int unpack_sha1_short_header(git_zstream *stream,\n@@ -2173,7 +2174,7 @@ int read_loose_object(const char *path,\n \n \t*contents = NULL;\n \n-\tmap = map_sha1_file_1(path, NULL, &mapsize);\n+\tmap = map_sha1_file_1(the_repository, path, NULL, &mapsize);\n \tif (!map) {\n \t\terror_errno(\"unable to mmap %s\", path);\n \t\tgoto out;\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339157","messageId":"20180213012241.187007-13-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 12/26] sha1_file: add repository argument to prepare_alt_odb","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:27Z","receivedAt":"2018-02-13T01:23:53Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"See previous patch for explanation.\n\nWhile at it, move the declaration to object-store.h,\nwhere it should be easier to find.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/fsck.c | 2 +-\n cache.h | 1 -\n object-store.h | 3 +++\n packfile.c | 2 +-\n sha1_file.c | 13 +++++++------\n sha1_name.c | 3 ++-\n 6 files changed, 14 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex d4d249c2ed..00ec8eecf0 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -695,7 +695,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t} else {\n \t\tfsck_object_dir(get_object_directory());\n \n-\t\tprepare_alt_odb();\n+\t\tprepare_alt_odb(the_repository);\n \t\tfor (alt = the_repository->objects.alt_odb_list;\n \t\t\t\talt; alt = alt->next)\n \t\t\tfsck_object_dir(alt->path);\ndiff --git a/cache.h b/cache.h\nindex 459fd01dbb..70518e24ce 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1592,7 +1592,6 @@ struct alternate_object_database {\n \n \tchar path[FLEX_ARRAY];\n };\n-extern void prepare_alt_odb(void);\n extern char *compute_alternate_path(const char *path, struct strbuf *err);\n typedef int alt_odb_fn(struct alternate_object_database *, void *);\n extern int foreach_alt_odb(alt_odb_fn, void*);\ndiff --git a/object-store.h b/object-store.h\nindex 86f1ec2cd0..d96a16edd1 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -61,4 +61,7 @@ struct packed_git {\n \tchar pack_name[FLEX_ARRAY]; /* more */\n };\n \n+#define prepare_alt_odb(r) prepare_alt_odb_##r()\n+extern void prepare_alt_odb_the_repository(void);\n+\n #endif /* OBJECT_STORE_H */\ndiff --git a/packfile.c b/packfile.c\nindex a53706cf16..9c49b34d2b 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -879,7 +879,7 @@ void prepare_packed_git(void)\n \tif (the_repository->objects.packed_git_initialized)\n \t\treturn;\n \tprepare_packed_git_one(get_object_directory(), 1);\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)\n \t\tprepare_packed_git_one(alt->path, 0);\n \trearrange_packed_git();\ndiff --git a/sha1_file.c b/sha1_file.c\nindex b090f403d8..d7f271fe6e 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -23,6 +23,7 @@\n #include \"sha1-lookup.h\"\n #include \"bulk-checkin.h\"\n #include \"repository.h\"\n+#include \"object-store.h\"\n #include \"streaming.h\"\n #include \"dir.h\"\n #include \"mru.h\"\n@@ -585,7 +586,7 @@ void add_to_alternates_memory(const char *reference)\n \t * Make sure alternates are initialized, or else our entry may be\n \t * overwritten when they are.\n \t */\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \n \tlink_alt_odb_entries(the_repository, reference,\n \t\t\t '\\n', NULL, 0);\n@@ -671,7 +672,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \tstruct alternate_object_database *ent;\n \tint r = 0;\n \n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \tfor (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {\n \t\tr = fn(ent, cb);\n \t\tif (r)\n@@ -680,7 +681,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \treturn r;\n }\n \n-void prepare_alt_odb(void)\n+void prepare_alt_odb_the_repository(void)\n {\n \tconst char *alt;\n \n@@ -729,7 +730,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)\n static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)\n {\n \tstruct alternate_object_database *alt;\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\tconst char *path = alt_sha1_path(alt, sha1);\n \t\tif (check_and_freshen_file(path, freshen))\n@@ -884,7 +885,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n \tif (!lstat(*path, st))\n \t\treturn 0;\n \n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \terrno = ENOENT;\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n@@ -911,7 +912,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \t\treturn fd;\n \tmost_interesting_errno = errno;\n \n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tfd = git_open(*path);\ndiff --git a/sha1_name.c b/sha1_name.c\nindex 016c883b5c..3e490ee8f6 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -11,6 +11,7 @@\n #include \"sha1-array.h\"\n #include \"packfile.h\"\n #include \"repository.h\"\n+#include \"object-store.h\"\n \n static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);\n \n@@ -353,7 +354,7 @@ static int init_object_disambiguation(const char *name, int len,\n \n \tds->len = len;\n \tds->hex_pfx[len] = '\\0';\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \treturn 0;\n }\n \n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339158","messageId":"20180213012241.187007-14-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 13/26] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:28Z","receivedAt":"2018-02-13T01:23:56Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Actually this also allows read_info_alternates and link_alt_odb_entry to\nhandle arbitrary repositories, but link_alt_odb_entries is the most\ninteresting function in this set of functions, hence the commit subject.\n\nThese functions span a strongly connected component in the function\ngraph, i.e. the recursive call chain might look like\n\n -> link_alt_odb_entries\n -> link_alt_odb_entry\n -> read_info_alternates\n -> link_alt_odb_entries\n\nThat is why we need to convert all these functions at the same time.\n\nSigned-off-by: Jonathan Nieder \nSigned-off-by: Stefan Beller \n---\n cache.h | 4 ++++\n sha1_file.c | 36 ++++++++++++++++--------------------\n 2 files changed, 20 insertions(+), 20 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 70518e24ce..09d37cb651 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1590,6 +1590,10 @@ struct alternate_object_database {\n \tchar loose_objects_subdir_seen[256];\n \tstruct oid_array loose_objects_cache;\n \n+\t/*\n+\t * Path to the alternative object store. If this is a relative path,\n+\t * it is relative to the current working directory.\n+\t */\n \tchar path[FLEX_ARRAY];\n };\n extern char *compute_alternate_path(const char *path, struct strbuf *err);\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d7f271fe6e..d18ce3aeba 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -394,11 +394,10 @@ static int alt_odb_usable(struct raw_object_store *o,\n * SHA1, an extra slash for the first level indirection, and the\n * terminating NUL.\n */\n-#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)\n-static void read_info_alternates_the_repository(const char *relative_base,\n-\t\t\t\t\t\tint depth);\n-#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)\n-static int link_alt_odb_entry_the_repository(const char *entry,\n+static void read_info_alternates(struct repository *r,\n+\t\t\t\t const char *relative_base,\n+\t\t\t\t int depth);\n+static int link_alt_odb_entry(struct repository *r, const char *entry,\n \tconst char *relative_base, int depth, const char *normalized_objdir)\n {\n \tstruct alternate_object_database *ent;\n@@ -424,7 +423,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,\n \twhile (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')\n \t\tstrbuf_setlen(&pathbuf, pathbuf.len - 1);\n \n-\tif (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {\n+\tif (!alt_odb_usable(&r->objects, &pathbuf, normalized_objdir)) {\n \t\tstrbuf_release(&pathbuf);\n \t\treturn -1;\n \t}\n@@ -432,12 +431,12 @@ static int link_alt_odb_entry_the_repository(const char *entry,\n \tent = alloc_alt_odb(pathbuf.buf);\n \n \t/* add the alternate entry */\n-\t*the_repository->objects.alt_odb_tail = ent;\n-\tthe_repository->objects.alt_odb_tail = &(ent->next);\n+\t*r->objects.alt_odb_tail = ent;\n+\tr->objects.alt_odb_tail = &(ent->next);\n \tent->next = NULL;\n \n \t/* recursively add alternates */\n-\tread_info_alternates(the_repository, pathbuf.buf, depth + 1);\n+\tread_info_alternates(r, pathbuf.buf, depth + 1);\n \n \tstrbuf_release(&pathbuf);\n \treturn 0;\n@@ -472,12 +471,8 @@ static const char *parse_alt_odb_entry(const char *string,\n \treturn end;\n }\n \n-#define link_alt_odb_entries(r, a, s, rb, d) \\\n-\tlink_alt_odb_entries_##r(a, s, rb, d)\n-static void link_alt_odb_entries_the_repository(const char *alt,\n-\t\t\t\t\t\tint sep,\n-\t\t\t\t\t\tconst char *relative_base,\n-\t\t\t\t\t\tint depth)\n+static void link_alt_odb_entries(struct repository *r, const char *alt,\n+\t\t\t\t int sep, const char *relative_base, int depth)\n {\n \tstruct strbuf objdirbuf = STRBUF_INIT;\n \tstruct strbuf entry = STRBUF_INIT;\n@@ -491,7 +486,7 @@ static void link_alt_odb_entries_the_repository(const char *alt,\n \t\treturn;\n \t}\n \n-\tstrbuf_add_absolute_path(&objdirbuf, get_object_directory());\n+\tstrbuf_add_absolute_path(&objdirbuf, r->objects.objectdir);\n \tif (strbuf_normalize_path(&objdirbuf) < 0)\n \t\tdie(\"unable to normalize object directory: %s\",\n \t\t objdirbuf.buf);\n@@ -500,15 +495,16 @@ static void link_alt_odb_entries_the_repository(const char *alt,\n \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n \t\tif (!entry.len)\n \t\t\tcontinue;\n-\t\tlink_alt_odb_entry(the_repository, entry.buf,\n+\t\tlink_alt_odb_entry(r, entry.buf,\n \t\t\t\t relative_base, depth, objdirbuf.buf);\n \t}\n \tstrbuf_release(&entry);\n \tstrbuf_release(&objdirbuf);\n }\n \n-static void read_info_alternates_the_repository(const char *relative_base,\n-\t\t\t\t\t\tint depth)\n+static void read_info_alternates(struct repository *r,\n+\t\t\t\t const char *relative_base,\n+\t\t\t\t int depth)\n {\n \tchar *path;\n \tstruct strbuf buf = STRBUF_INIT;\n@@ -520,7 +516,7 @@ static void read_info_alternates_the_repository(const char *relative_base,\n \t\treturn;\n \t}\n \n-\tlink_alt_odb_entries(the_repository, buf.buf, '\\n', relative_base, depth);\n+\tlink_alt_odb_entries(r, buf.buf, '\\n', relative_base, depth);\n \tstrbuf_release(&buf);\n \tfree(path);\n }\n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339159","messageId":"20180213012241.187007-7-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 06/26] pack: move prepare_packed_git_run_once to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:21Z","receivedAt":"2018-02-13T01:24:00Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Jonathan Nieder \n\nEach repository's object store can be initialized independently, so\nthey must not share a run_once variable.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 8 +++++++-\n packfile.c | 7 +++----\n 2 files changed, 10 insertions(+), 5 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 8778ab9685..9fd109fa9c 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -20,8 +20,14 @@ struct raw_object_store {\n \n \tstruct alternate_object_database *alt_odb_list;\n \tstruct alternate_object_database **alt_odb_tail;\n+\n+\t/*\n+\t * Whether packed_git has already been populated with this repository's\n+\t * packs.\n+\t */\n+\tunsigned packed_git_initialized : 1;\n };\n-#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_INIT, NULL, NULL }\n+#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_INIT, NULL, NULL, 0 }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/packfile.c b/packfile.c\nindex 2ce1b2cef7..c107fce171 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -873,12 +873,11 @@ static void prepare_packed_git_mru(void)\n \t\tmru_append(&the_repository->objects.packed_git_mru, p);\n }\n \n-static int prepare_packed_git_run_once = 0;\n void prepare_packed_git(void)\n {\n \tstruct alternate_object_database *alt;\n \n-\tif (prepare_packed_git_run_once)\n+\tif (the_repository->objects.packed_git_initialized)\n \t\treturn;\n \tprepare_packed_git_one(get_object_directory(), 1);\n \tprepare_alt_odb();\n@@ -886,13 +885,13 @@ void prepare_packed_git(void)\n \t\tprepare_packed_git_one(alt->path, 0);\n \trearrange_packed_git();\n \tprepare_packed_git_mru();\n-\tprepare_packed_git_run_once = 1;\n+\tthe_repository->objects.packed_git_initialized = 1;\n }\n \n void reprepare_packed_git(void)\n {\n \tapproximate_object_count_valid = 0;\n-\tprepare_packed_git_run_once = 0;\n+\tthe_repository->objects.packed_git_initialized = 0;\n \tprepare_packed_git();\n }\n \n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339160","messageId":"20180213012241.187007-5-sbeller@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"[PATCH 04/26] object-store: move packed_git and packed_git_mru to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:22:19Z","receivedAt":"2018-02-13T01:24:06Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a process with multiple repositories open, packfile accessors\nshould be associated to a single repository and not shared globally.\nMove packed_git and packed_git_mru into the_repository and adjust\ncallers to reflect this.\n\nPatch generated by\n\n 1. Moving the struct packed_git declaration to object-store.h\n and packed_git, packed_git_mru globals to struct object_store.\n\n 2. Applying the semantic patch\n contrib/coccinelle/refactoring/packed_git.cocci to adjust callers.\n This semantic patch is placed in a sub directory of the coccinelle\n contrib dir, as this semantic patch is not expected to be of general\n usefulness; it is only useful during developing this series and\n merging it with other topics in flight. At a later date, just\n delete that semantic patch.\n\n 3. Applying line wrapping fixes from \"make style\" to break the\n resulting long lines.\n\n 4. Adding missing #includes of repository.h and object-store.h\n where needed.\n\n 5. As the packfiles are now owned by an objectstore/repository, which\n is ephemeral unlike globals, we introduce memory leaks. So address\n them in raw_object_store_clear().\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/count-objects.c | 6 ++--\n builtin/fsck.c | 7 +++--\n builtin/gc.c | 4 ++-\n builtin/index-pack.c | 1 +\n builtin/pack-objects.c | 21 ++++++++-----\n builtin/pack-redundant.c | 6 ++--\n builtin/receive-pack.c | 1 +\n cache.h | 28 ------------------\n contrib/coccinelle/refactoring/packed_git.cocci | 7 +++++\n fast-import.c | 6 ++--\n http-backend.c | 6 ++--\n http-push.c | 1 +\n http-walker.c | 1 +\n http.c | 1 +\n mru.h | 1 +\n object-store.h | 32 +++++++++++++++++++-\n object.c | 6 ++++\n pack-bitmap.c | 4 ++-\n pack-check.c | 1 +\n pack-revindex.c | 1 +\n packfile.c | 39 +++++++++++++------------\n reachable.c | 1 +\n server-info.c | 6 ++--\n sha1_name.c | 5 ++--\n 24 files changed, 120 insertions(+), 72 deletions(-)\n create mode 100644 contrib/coccinelle/refactoring/packed_git.cocci\n\ndiff --git a/builtin/count-objects.c b/builtin/count-objects.c\nindex 33343818c8..9334648dcc 100644\n--- a/builtin/count-objects.c\n+++ b/builtin/count-objects.c\n@@ -7,6 +7,8 @@\n #include \"cache.h\"\n #include \"config.h\"\n #include \"dir.h\"\n+#include \"repository.h\"\n+#include \"object-store.h\"\n #include \"builtin.h\"\n #include \"parse-options.h\"\n #include \"quote.h\"\n@@ -120,9 +122,9 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)\n \t\tstruct strbuf loose_buf = STRBUF_INIT;\n \t\tstruct strbuf pack_buf = STRBUF_INIT;\n \t\tstruct strbuf garbage_buf = STRBUF_INIT;\n-\t\tif (!packed_git)\n+\t\tif (!the_repository->objects.packed_git)\n \t\t\tprepare_packed_git();\n-\t\tfor (p = packed_git; p; p = p->next) {\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t\tif (!p->pack_local)\n \t\t\t\tcontinue;\n \t\t\tif (open_pack_index(p))\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 1048255da1..d4d249c2ed 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -2,6 +2,7 @@\n #include \"cache.h\"\n #include \"repository.h\"\n #include \"config.h\"\n+#include \"object-store.h\"\n #include \"commit.h\"\n #include \"tree.h\"\n #include \"blob.h\"\n@@ -707,7 +708,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t\t\tprepare_packed_git();\n \n \t\t\tif (show_progress) {\n-\t\t\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\t\tfor (p = the_repository->objects.packed_git; p;\n+\t\t\t\t p = p->next) {\n \t\t\t\t\tif (open_pack_index(p))\n \t\t\t\t\t\tcontinue;\n \t\t\t\t\ttotal += p->num_objects;\n@@ -715,7 +717,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \n \t\t\t\tprogress = start_progress(_(\"Checking objects\"), total);\n \t\t\t}\n-\t\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\tfor (p = the_repository->objects.packed_git; p;\n+\t\t\t p = p->next) {\n \t\t\t\t/* verify gives error messages itself */\n \t\t\t\tif (verify_pack(p, fsck_obj_buffer,\n \t\t\t\t\t\tprogress, count))\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 3c5eae0edf..6970f325e5 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -11,6 +11,7 @@\n */\n \n #include \"builtin.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n #include \"tempfile.h\"\n #include \"lockfile.h\"\n@@ -18,6 +19,7 @@\n #include \"run-command.h\"\n #include \"sigchain.h\"\n #include \"argv-array.h\"\n+#include \"object-store.h\"\n #include \"commit.h\"\n #include \"packfile.h\"\n \n@@ -173,7 +175,7 @@ static int too_many_packs(void)\n \t\treturn 0;\n \n \tprepare_packed_git();\n-\tfor (cnt = 0, p = packed_git; p; p = p->next) {\n+\tfor (cnt = 0, p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (!p->pack_local)\n \t\t\tcontinue;\n \t\tif (p->pack_keep)\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 4c51aec81f..1902d4e096 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -12,6 +12,7 @@\n #include \"exec_cmd.h\"\n #include \"streaming.h\"\n #include \"thread-utils.h\"\n+#include \"object-store.h\"\n #include \"packfile.h\"\n \n static const char index_pack_usage[] =\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 6b9cfc289d..2ee8b69238 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1,6 +1,8 @@\n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n+#include \"object-store.h\"\n #include \"attr.h\"\n #include \"object.h\"\n #include \"blob.h\"\n@@ -1023,7 +1025,8 @@ static int want_object_in_pack(const struct object_id *oid,\n \t\t\treturn want;\n \t}\n \n-\tfor (entry = packed_git_mru.head; entry; entry = entry->next) {\n+\tfor (entry = the_repository->objects.packed_git_mru.head; entry;\n+\t entry = entry->next) {\n \t\tstruct packed_git *p = entry->item;\n \t\toff_t offset;\n \n@@ -1041,7 +1044,8 @@ static int want_object_in_pack(const struct object_id *oid,\n \t\t\t}\n \t\t\twant = want_found_object(exclude, p);\n \t\t\tif (!exclude && want > 0)\n-\t\t\t\tmru_mark(&packed_git_mru, entry);\n+\t\t\t\tmru_mark(&the_repository->objects.packed_git_mru,\n+\t\t\t\t\t entry);\n \t\t\tif (want != -1)\n \t\t\t\treturn want;\n \t\t}\n@@ -2648,7 +2652,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)\n \n \tmemset(&in_pack, 0, sizeof(in_pack));\n \n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tstruct object_id oid;\n \t\tstruct object *o;\n \n@@ -2711,7 +2715,8 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)\n \tstatic struct packed_git *last_found = (void *)1;\n \tstruct packed_git *p;\n \n-\tp = (last_found != (void *)1) ? last_found : packed_git;\n+\tp = (last_found != (void *)1) ? last_found :\n+\t\t\t\t\tthe_repository->objects.packed_git;\n \n \twhile (p) {\n \t\tif ((!p->pack_local || p->pack_keep) &&\n@@ -2720,7 +2725,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)\n \t\t\treturn 1;\n \t\t}\n \t\tif (p == last_found)\n-\t\t\tp = packed_git;\n+\t\t\tp = the_repository->objects.packed_git;\n \t\telse\n \t\t\tp = p->next;\n \t\tif (p == last_found)\n@@ -2756,7 +2761,7 @@ static void loosen_unused_packed_objects(struct rev_info *revs)\n \tuint32_t i;\n \tstruct object_id oid;\n \n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (!p->pack_local || p->pack_keep)\n \t\t\tcontinue;\n \n@@ -3119,7 +3124,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \tprepare_packed_git();\n \tif (ignore_packed_keep) {\n \t\tstruct packed_git *p;\n-\t\tfor (p = packed_git; p; p = p->next)\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\t\tif (p->pack_local && p->pack_keep)\n \t\t\t\tbreak;\n \t\tif (!p) /* no keep-able packs found */\n@@ -3132,7 +3137,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \t\t * also covers non-local objects\n \t\t */\n \t\tstruct packed_git *p;\n-\t\tfor (p = packed_git; p; p = p->next) {\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t\tif (!p->pack_local) {\n \t\t\t\thave_non_local_packs = 1;\n \t\t\t\tbreak;\ndiff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c\nindex aaa8136322..55462bc826 100644\n--- a/builtin/pack-redundant.c\n+++ b/builtin/pack-redundant.c\n@@ -7,6 +7,8 @@\n */\n \n #include \"builtin.h\"\n+#include \"repository.h\"\n+#include \"object-store.h\"\n #include \"packfile.h\"\n \n #define BLKSIZE 512\n@@ -571,7 +573,7 @@ static struct pack_list * add_pack(struct packed_git *p)\n \n static struct pack_list * add_pack_file(const char *filename)\n {\n-\tstruct packed_git *p = packed_git;\n+\tstruct packed_git *p = the_repository->objects.packed_git;\n \n \tif (strlen(filename) < 40)\n \t\tdie(\"Bad pack filename: %s\", filename);\n@@ -586,7 +588,7 @@ static struct pack_list * add_pack_file(const char *filename)\n \n static void load_all(void)\n {\n-\tstruct packed_git *p = packed_git;\n+\tstruct packed_git *p = the_repository->objects.packed_git;\n \n \twhile (p) {\n \t\tadd_pack(p);\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex b7ce7c7f52..b2eac79a6e 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -7,6 +7,7 @@\n #include \"sideband.h\"\n #include \"run-command.h\"\n #include \"exec_cmd.h\"\n+#include \"object-store.h\"\n #include \"commit.h\"\n #include \"object.h\"\n #include \"remote.h\"\ndiff --git a/cache.h b/cache.h\nindex 918c2f15b4..459fd01dbb 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1633,34 +1633,6 @@ struct pack_window {\n \tunsigned int inuse_cnt;\n };\n \n-extern struct packed_git {\n-\tstruct packed_git *next;\n-\tstruct pack_window *windows;\n-\toff_t pack_size;\n-\tconst void *index_data;\n-\tsize_t index_size;\n-\tuint32_t num_objects;\n-\tuint32_t num_bad_objects;\n-\tunsigned char *bad_object_sha1;\n-\tint index_version;\n-\ttime_t mtime;\n-\tint pack_fd;\n-\tunsigned pack_local:1,\n-\t\t pack_keep:1,\n-\t\t freshened:1,\n-\t\t do_not_close:1;\n-\tunsigned char sha1[20];\n-\tstruct revindex_entry *revindex;\n-\t/* something like \".git/objects/pack/xxxxx.pack\" */\n-\tchar pack_name[FLEX_ARRAY]; /* more */\n-} *packed_git;\n-\n-/*\n- * A most-recently-used ordered version of the packed_git list, which can\n- * be iterated instead of packed_git (and marked via mru_mark).\n- */\n-extern struct mru packed_git_mru;\n-\n struct pack_entry {\n \toff_t offset;\n \tunsigned char sha1[20];\ndiff --git a/contrib/coccinelle/refactoring/packed_git.cocci b/contrib/coccinelle/refactoring/packed_git.cocci\nnew file mode 100644\nindex 0000000000..da317a51a9\n--- /dev/null\n+++ b/contrib/coccinelle/refactoring/packed_git.cocci\n@@ -0,0 +1,7 @@\n+@@ @@\n+- packed_git\n++ the_repository->objects.packed_git\n+\n+@@ @@\n+- packed_git_mru\n++ the_repository->objects.packed_git_mru\ndiff --git a/fast-import.c b/fast-import.c\nindex b70ac025e0..1685fe59a2 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -154,8 +154,10 @@ Format of STDIN stream:\n \n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n #include \"lockfile.h\"\n+#include \"object-store.h\"\n #include \"object.h\"\n #include \"blob.h\"\n #include \"tree.h\"\n@@ -1110,7 +1112,7 @@ static int store_object(\n \tif (e->idx.offset) {\n \t\tduplicate_count_by_type[type]++;\n \t\treturn 1;\n-\t} else if (find_sha1_pack(oid.hash, packed_git)) {\n+\t} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {\n \t\te->type = type;\n \t\te->pack_id = MAX_PACK_ID;\n \t\te->idx.offset = 1; /* just not zero! */\n@@ -1305,7 +1307,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)\n \t\tduplicate_count_by_type[OBJ_BLOB]++;\n \t\ttruncate_pack(&checkpoint);\n \n-\t} else if (find_sha1_pack(oid.hash, packed_git)) {\n+\t} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {\n \t\te->type = OBJ_BLOB;\n \t\te->pack_id = MAX_PACK_ID;\n \t\te->idx.offset = 1; /* just not zero! */\ndiff --git a/http-backend.c b/http-backend.c\nindex f3dc218b2a..4d93126c15 100644\n--- a/http-backend.c\n+++ b/http-backend.c\n@@ -1,7 +1,9 @@\n #include \"cache.h\"\n #include \"config.h\"\n+#include \"repository.h\"\n #include \"refs.h\"\n #include \"pkt-line.h\"\n+#include \"object-store.h\"\n #include \"object.h\"\n #include \"tag.h\"\n #include \"exec_cmd.h\"\n@@ -518,13 +520,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg)\n \n \tselect_getanyfile(hdr);\n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (p->pack_local)\n \t\t\tcnt++;\n \t}\n \n \tstrbuf_grow(&buf, cnt * 53 + 2);\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (p->pack_local)\n \t\t\tstrbuf_addf(&buf, \"P %s\\n\", p->pack_name + objdirlen + 6);\n \t}\ndiff --git a/http-push.c b/http-push.c\nindex 14435ab65d..34c5d030b7 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -1,4 +1,5 @@\n #include \"cache.h\"\n+#include \"object-store.h\"\n #include \"commit.h\"\n #include \"tag.h\"\n #include \"blob.h\"\ndiff --git a/http-walker.c b/http-walker.c\nindex 1ae8363de2..a1c6f2639b 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -4,6 +4,7 @@\n #include \"http.h\"\n #include \"list.h\"\n #include \"transport.h\"\n+#include \"object-store.h\"\n #include \"packfile.h\"\n \n struct alt_base {\ndiff --git a/http.c b/http.c\nindex 5977712712..ab989b88dd 100644\n--- a/http.c\n+++ b/http.c\n@@ -1,6 +1,7 @@\n #include \"git-compat-util.h\"\n #include \"http.h\"\n #include \"config.h\"\n+#include \"object-store.h\"\n #include \"pack.h\"\n #include \"sideband.h\"\n #include \"run-command.h\"\ndiff --git a/mru.h b/mru.h\nindex 42e4aeaa10..d7de81efb9 100644\n--- a/mru.h\n+++ b/mru.h\n@@ -37,6 +37,7 @@ struct mru_entry {\n struct mru {\n \tstruct mru_entry *head, *tail;\n };\n+#define MRU_INIT { NULL, NULL }\n \n void mru_append(struct mru *mru, void *item);\n void mru_mark(struct mru *mru, struct mru_entry *entry);\ndiff --git a/object-store.h b/object-store.h\nindex 44b8f84753..8778ab9685 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -2,6 +2,7 @@\n #define OBJECT_STORE_H\n \n #include \"cache.h\"\n+#include \"mru.h\"\n \n struct raw_object_store {\n \t/*\n@@ -10,11 +11,40 @@ struct raw_object_store {\n \t */\n \tchar *objectdir;\n \n+\tstruct packed_git *packed_git;\n+\t/*\n+\t * A most-recently-used ordered version of the packed_git list, which can\n+\t * be iterated instead of packed_git (and marked via mru_mark).\n+\t */\n+\tstruct mru packed_git_mru;\n+\n \tstruct alternate_object_database *alt_odb_list;\n \tstruct alternate_object_database **alt_odb_tail;\n };\n-#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }\n+#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_INIT, NULL, NULL }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \n+struct packed_git {\n+\tstruct packed_git *next;\n+\tstruct pack_window *windows;\n+\toff_t pack_size;\n+\tconst void *index_data;\n+\tsize_t index_size;\n+\tuint32_t num_objects;\n+\tuint32_t num_bad_objects;\n+\tunsigned char *bad_object_sha1;\n+\tint index_version;\n+\ttime_t mtime;\n+\tint pack_fd;\n+\tunsigned pack_local:1,\n+\t\t pack_keep:1,\n+\t\t freshened:1,\n+\t\t do_not_close:1;\n+\tunsigned char sha1[20];\n+\tstruct revindex_entry *revindex;\n+\t/* something like \".git/objects/pack/xxxxx.pack\" */\n+\tchar pack_name[FLEX_ARRAY]; /* more */\n+};\n+\n #endif /* OBJECT_STORE_H */\ndiff --git a/object.c b/object.c\nindex d778f09717..81d083bec1 100644\n--- a/object.c\n+++ b/object.c\n@@ -466,4 +466,10 @@ void raw_object_store_clear(struct raw_object_store *o)\n \n \tfree_alt_odbs(o);\n \to->alt_odb_tail = NULL;\n+\n+\tmru_clear(&o->packed_git_mru);\n+\t/*\n+\t * TODO: call close_all_packs once migrated to\n+\t * take an object store argument\n+\t */\n }\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9270983e5f..d51172b1d5 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -5,11 +5,13 @@\n #include \"revision.h\"\n #include \"progress.h\"\n #include \"list-objects.h\"\n+#include \"object-store.h\"\n #include \"pack.h\"\n #include \"pack-bitmap.h\"\n #include \"pack-revindex.h\"\n #include \"pack-objects.h\"\n #include \"packfile.h\"\n+#include \"repository.h\"\n \n /*\n * An entry on the bitmap index, representing the bitmap for a given\n@@ -335,7 +337,7 @@ static int open_pack_bitmap(void)\n \tassert(!bitmap_git.map && !bitmap_git.loaded);\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (open_pack_bitmap_1(p) == 0)\n \t\t\tret = 0;\n \t}\ndiff --git a/pack-check.c b/pack-check.c\nindex 073c1fbd46..c80c3ed8fd 100644\n--- a/pack-check.c\n+++ b/pack-check.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"pack.h\"\n+#include \"object-store.h\"\n #include \"pack-revindex.h\"\n #include \"progress.h\"\n #include \"packfile.h\"\ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex 1b7ebd8d7e..82c3119652 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"pack-revindex.h\"\n+#include \"object-store.h\"\n \n /*\n * Pack index for existing packs give us easy access to the offsets into\ndiff --git a/packfile.c b/packfile.c\nindex d61076faaf..11b0e6613c 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -7,6 +7,7 @@\n #include \"packfile.h\"\n #include \"delta.h\"\n #include \"list.h\"\n+#include \"object-store.h\"\n #include \"streaming.h\"\n #include \"sha1-lookup.h\"\n \n@@ -40,8 +41,6 @@ static unsigned int pack_open_fds;\n static unsigned int pack_max_fds;\n static size_t peak_pack_mapped;\n static size_t pack_mapped;\n-struct packed_git *packed_git;\n-struct mru packed_git_mru;\n \n #define SZ_FMT PRIuMAX\n static inline uintmax_t sz_fmt(size_t s) { return s; }\n@@ -241,7 +240,7 @@ static int unuse_one_window(struct packed_git *current)\n \n \tif (current)\n \t\tscan_windows(current, &lru_p, &lru_w, &lru_l);\n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tscan_windows(p, &lru_p, &lru_w, &lru_l);\n \tif (lru_p) {\n \t\tmunmap(lru_w->base, lru_w->len);\n@@ -311,7 +310,7 @@ void close_all_packs(void)\n {\n \tstruct packed_git *p;\n \n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tif (p->do_not_close)\n \t\t\tdie(\"BUG: want to close pack marked 'do-not-close'\");\n \t\telse\n@@ -379,7 +378,7 @@ static int close_one_pack(void)\n \tstruct pack_window *mru_w = NULL;\n \tint accept_windows_inuse = 1;\n \n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (p->pack_fd == -1)\n \t\t\tcontinue;\n \t\tfind_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);\n@@ -677,8 +676,8 @@ void install_packed_git(struct packed_git *pack)\n \tif (pack->pack_fd != -1)\n \t\tpack_open_fds++;\n \n-\tpack->next = packed_git;\n-\tpacked_git = pack;\n+\tpack->next = the_repository->objects.packed_git;\n+\tthe_repository->objects.packed_git = pack;\n }\n \n void (*report_garbage)(unsigned seen_bits, const char *path);\n@@ -760,7 +759,8 @@ static void prepare_packed_git_one(char *objdir, int local)\n \t\tbase_len = path.len;\n \t\tif (strip_suffix_mem(path.buf, &base_len, \".idx\")) {\n \t\t\t/* Don't reopen a pack we already have. */\n-\t\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\tfor (p = the_repository->objects.packed_git; p;\n+\t\t\t p = p->next) {\n \t\t\t\tsize_t len;\n \t\t\t\tif (strip_suffix(p->pack_name, \".pack\", &len) &&\n \t\t\t\t len == base_len &&\n@@ -810,7 +810,7 @@ unsigned long approximate_object_count(void)\n \n \t\tprepare_packed_git();\n \t\tcount = 0;\n-\t\tfor (p = packed_git; p; p = p->next) {\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t\tif (open_pack_index(p))\n \t\t\t\tcontinue;\n \t\t\tcount += p->num_objects;\n@@ -859,17 +859,18 @@ static int sort_pack(const void *a_, const void *b_)\n \n static void rearrange_packed_git(void)\n {\n-\tpacked_git = llist_mergesort(packed_git, get_next_packed_git,\n-\t\t\t\t set_next_packed_git, sort_pack);\n+\tthe_repository->objects.packed_git = llist_mergesort(\n+\t\tthe_repository->objects.packed_git, get_next_packed_git,\n+\t\tset_next_packed_git, sort_pack);\n }\n \n static void prepare_packed_git_mru(void)\n {\n \tstruct packed_git *p;\n \n-\tmru_clear(&packed_git_mru);\n-\tfor (p = packed_git; p; p = p->next)\n-\t\tmru_append(&packed_git_mru, p);\n+\tmru_clear(&the_repository->objects.packed_git_mru);\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n+\t\tmru_append(&the_repository->objects.packed_git_mru, p);\n }\n \n static int prepare_packed_git_run_once = 0;\n@@ -1003,7 +1004,7 @@ const struct packed_git *has_packed_and_bad(const unsigned char *sha1)\n \tstruct packed_git *p;\n \tunsigned i;\n \n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tfor (i = 0; i < p->num_bad_objects; i++)\n \t\t\tif (!hashcmp(sha1, p->bad_object_sha1 + 20 * i))\n \t\t\t\treturn p;\n@@ -1835,12 +1836,12 @@ int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)\n \tstruct mru_entry *p;\n \n \tprepare_packed_git();\n-\tif (!packed_git)\n+\tif (!the_repository->objects.packed_git)\n \t\treturn 0;\n \n-\tfor (p = packed_git_mru.head; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git_mru.head; p; p = p->next) {\n \t\tif (fill_pack_entry(sha1, e, p->item)) {\n-\t\t\tmru_mark(&packed_git_mru, p);\n+\t\t\tmru_mark(&the_repository->objects.packed_git_mru, p);\n \t\t\treturn 1;\n \t\t}\n \t}\n@@ -1887,7 +1888,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)\n \tint pack_errors = 0;\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)\n \t\t\tcontinue;\n \t\tif (open_pack_index(p)) {\ndiff --git a/reachable.c b/reachable.c\nindex 88d7d679da..59ed16ba76 100644\n--- a/reachable.c\n+++ b/reachable.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"refs.h\"\n+#include \"object-store.h\"\n #include \"tag.h\"\n #include \"commit.h\"\n #include \"blob.h\"\ndiff --git a/server-info.c b/server-info.c\nindex 26a6c20b7d..96885c45f6 100644\n--- a/server-info.c\n+++ b/server-info.c\n@@ -1,5 +1,7 @@\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"refs.h\"\n+#include \"object-store.h\"\n #include \"object.h\"\n #include \"commit.h\"\n #include \"tag.h\"\n@@ -200,7 +202,7 @@ static void init_pack_info(const char *infofile, int force)\n \tobjdirlen = strlen(objdir);\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t/* we ignore things on alternate path since they are\n \t\t * not available to the pullers in general.\n \t\t */\n@@ -210,7 +212,7 @@ static void init_pack_info(const char *infofile, int force)\n \t}\n \tnum_pack = i;\n \tinfo = xcalloc(num_pack, sizeof(struct pack_info *));\n-\tfor (i = 0, p = packed_git; p; p = p->next) {\n+\tfor (i = 0, p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (!p->pack_local)\n \t\t\tcontinue;\n \t\tinfo[i] = xcalloc(1, sizeof(struct pack_info));\ndiff --git a/sha1_name.c b/sha1_name.c\nindex 957ce25680..016c883b5c 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -196,7 +196,8 @@ static void find_short_packed_object(struct disambiguate_state *ds)\n \tstruct packed_git *p;\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p && !ds->ambiguous; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p && !ds->ambiguous;\n+\t p = p->next)\n \t\tunique_in_pack(p, ds);\n }\n \n@@ -566,7 +567,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)\n \tstruct packed_git *p;\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tfind_abbrev_len_for_pack(p, mad);\n }\n \n-- \n2.16.1.73.ga2c3e9663f.dirty\n\n"},{"id":"339161","messageId":"CAGZ79kY8XmoZHXkMBENJLAXdNNQr-+BTW7kb6BUTiMWxMR5+uw@mail.gmail.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T01:38:28Z","receivedAt":"2018-02-13T01:38:34Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Feb 12, 2018 at 5:22 PM, Stefan Beller wrote:\n\n> I developed this series [...] on top of current master.\n\nalso available at https://github.com/stefanbeller/git/tree/object-store-part1\n"},{"id":"339189","messageId":"20180213114903.GA21015@ash","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-13T11:49:04Z","receivedAt":"2018-02-13T11:49:15Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Mon, Feb 12, 2018 at 05:22:15PM -0800, Stefan Beller wrote:\n> This is a real take on the first part of the recent RFC[1].\n>\n> ...\n> \n> Duy suggested that we shall not use the repository blindly, but\n> should carefully examine whether to pass on an object store or the\n> refstore or such[4], which I agree with if it makes sense. This\n> series unfortunately has an issue with that as I would not want to\n> pass down the `ignore_env` flag separately from the object store, so\n> I made all functions that only take the object store to have the raw\n> object store as the first parameter, and others using the full\n> repository.\n\nSecond proposal :) How about you store ignore_env in raw_object_store?\nThis would not be the first time an object has some configuration\npassed in at construction time. And it has a \"constructor\" now,\nraw_object_store_init() (I probably should merge _setup in it too)\n\nThe core changes look like this. I have a full commit on top of your\nseries [1] that keeps sha1_file.c functions take 'struct\nraw_object_store' instead.\n\n[1] https://github.com/pclouds/git/tree/object-store-part1\n\n-- 8< --\ndiff --git a/object-store.h b/object-store.h\nindex f164c4e5c9..a8bf1738f2 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -34,9 +34,13 @@ struct raw_object_store {\n \t * packs.\n \t */\n \tunsigned packed_git_initialized : 1;\n+\n+\tunsigned ignore_env : 1;\n };\n #define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_INIT, NULL, NULL, 0, 0, 0 }\n \n+void raw_object_store_init(struct raw_object_store *ros, struct repository *r);\n+void raw_object_store_setup(struct raw_object_store *ros, char *);\n void raw_object_store_clear(struct raw_object_store *o);\n \n struct packed_git {\ndiff --git a/object.c b/object.c\nindex 79c2c447bc..a4534bf4c4 100644\n--- a/object.c\n+++ b/object.c\n@@ -461,6 +461,20 @@ static void free_alt_odbs(struct raw_object_store *o)\n \t}\n }\n \n+void raw_object_store_init(struct raw_object_store *o,\n+\t\t\t struct repository *r)\n+{\n+\t/* FIXME: memset? */\n+\to->ignore_env = r->ignore_env;\n+}\n+\n+void raw_object_store_setup(struct raw_object_store *o,\n+\t\t\t char *objectdir)\n+{\n+\tfree(o->objectdir);\n+\to->objectdir = objectdir;\n+}\n+\n void raw_object_store_clear(struct raw_object_store *o)\n {\n \tfree(o->objectdir);\ndiff --git a/repository.c b/repository.c\nindex bd2ad578de..ac5863d7e1 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -49,9 +49,9 @@ static void repo_setup_env(struct repository *repo)\n \t\t\t\t\t\t !repo->ignore_env);\n \tfree(repo->commondir);\n \trepo->commondir = strbuf_detach(&sb, NULL);\n-\tfree(repo->objects.objectdir);\n-\trepo->objects.objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,\n-\t\t\t\t\t \"objects\", !repo->ignore_env);\n+\traw_object_store_setup(&repo->objects,\n+\t\t\t git_path_from_env(DB_ENVIRONMENT, repo->commondir,\n+\t\t\t\t\t\t \"objects\", !repo->ignore_env));\n \tfree(repo->graft_file);\n \trepo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,\n \t\t\t\t\t \"info/grafts\", !repo->ignore_env);\n@@ -142,6 +142,8 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)\n \n \trepo->ignore_env = 1;\n \n+\traw_object_store_init(&repo->objects, repo);\n+\n \tif (repo_init_gitdir(repo, gitdir))\n \t\tgoto error;\n \ndiff --git a/sha1_file.c b/sha1_file.c\nindex c3f35914ce..3be58651a1 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -677,14 +677,14 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \treturn r;\n }\n \n-void prepare_alt_odb(struct repository *r)\n+void prepare_alt_odb(struct raw_object_store *ros)\n {\n \tif (r->objects.alt_odb_tail)\n \t\treturn;\n \n \tr->objects.alt_odb_tail = &r->objects.alt_odb_list;\n \n-\tif (!r->ignore_env) {\n+\tif (!ros->ignore_env) {\n \t\tconst char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);\n \t\tif (!alt)\n \t\t\talt = \"\";\n-- 8< --\n\n\n--\nDuy\n"},{"id":"339190","messageId":"CACsJy8C8+VnjrhEh3o36GUttZXmfiKxqjzS=2LzmfZKQjogojQ@mail.gmail.com","threadId":"47823","inReplyTo":"20180213114903.GA21015@ash","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-13T12:13:15Z","receivedAt":"2018-02-13T12:13:52Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Feb 13, 2018 at 6:49 PM, Duy Nguyen wrote:\n> On Mon, Feb 12, 2018 at 05:22:15PM -0800, Stefan Beller wrote:\n>> This is a real take on the first part of the recent RFC[1].\n>>\n>> ...\n>>\n>> Duy suggested that we shall not use the repository blindly, but\n>> should carefully examine whether to pass on an object store or the\n>> refstore or such[4], which I agree with if it makes sense. This\n>> series unfortunately has an issue with that as I would not want to\n>> pass down the `ignore_env` flag separately from the object store, so\n>> I made all functions that only take the object store to have the raw\n>> object store as the first parameter, and others using the full\n>> repository.\n>\n> Second proposal :) How about you store ignore_env in raw_object_store?\n> This would not be the first time an object has some configuration\n> passed in at construction time. And it has a \"constructor\" now,\n> raw_object_store_init() (I probably should merge _setup in it too)\n\nA bit more on this configuration parameters. Down the road I think we\nneed something like this anyway to delete global config vars like\npacked_git_window_size, delta_base_cache_limit... Either all these\nend up in raw_object_store, or raw_object_store holds a link to\n\"struct config_set\".\n\nThe ignore_env specifically though looks to me like a stop gap\nsolution until everything goes through repo_init() first. At that\npoint we don't have to delay getenv() anymore. We can getenv() all at\nrepo_init() then pass them in raw_object_store and ignore_env should\nbe gone. So sticking it inside raw_object_store _temporarily_ does not\nsound so bad.\n-- \nDuy\n"},{"id":"339202","messageId":"20180213165249.GB209601@google.com","threadId":"47823","inReplyTo":"CACsJy8C8+VnjrhEh3o36GUttZXmfiKxqjzS=2LzmfZKQjogojQ@mail.gmail.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-13T16:52:49Z","receivedAt":"2018-02-13T16:52:57Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/13, Duy Nguyen wrote:\n> On Tue, Feb 13, 2018 at 6:49 PM, Duy Nguyen wrote:\n> > On Mon, Feb 12, 2018 at 05:22:15PM -0800, Stefan Beller wrote:\n> >> This is a real take on the first part of the recent RFC[1].\n> >>\n> >> ...\n> >>\n> >> Duy suggested that we shall not use the repository blindly, but\n> >> should carefully examine whether to pass on an object store or the\n> >> refstore or such[4], which I agree with if it makes sense. This\n> >> series unfortunately has an issue with that as I would not want to\n> >> pass down the `ignore_env` flag separately from the object store, so\n> >> I made all functions that only take the object store to have the raw\n> >> object store as the first parameter, and others using the full\n> >> repository.\n> >\n> > Second proposal :) How about you store ignore_env in raw_object_store?\n> > This would not be the first time an object has some configuration\n> > passed in at construction time. And it has a \"constructor\" now,\n> > raw_object_store_init() (I probably should merge _setup in it too)\n> \n> A bit more on this configuration parameters. Down the road I think we\n> need something like this anyway to delete global config vars like\n> packed_git_window_size, delta_base_cache_limit... Either all these\n> end up in raw_object_store, or raw_object_store holds a link to\n> \"struct config_set\".\n> \n> The ignore_env specifically though looks to me like a stop gap\n> solution until everything goes through repo_init() first. At that\n> point we don't have to delay getenv() anymore. We can getenv() all at\n> repo_init() then pass them in raw_object_store and ignore_env should\n> be gone. So sticking it inside raw_object_store _temporarily_ does not\n> sound so bad.\n\nI like this approach, I mean at the moment we are replicating a single\nbit of data but that allows us to be able to limit the scope of where a\nrepository struct is passed, giving us a better abstraction layer.\n\n> -- \n> Duy\n\n-- \nBrandon Williams\n"},{"id":"339208","messageId":"CAGZ79kbmAkN5EpW1hzW-FFdR74c-qXJc5aKZX0ff9bU0gxMRqQ@mail.gmail.com","threadId":"47823","inReplyTo":"CACsJy8C8+VnjrhEh3o36GUttZXmfiKxqjzS=2LzmfZKQjogojQ@mail.gmail.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T17:47:38Z","receivedAt":"2018-02-13T17:47:45Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Feb 13, 2018 at 4:13 AM, Duy Nguyen wrote:\n> On Tue, Feb 13, 2018 at 6:49 PM, Duy Nguyen wrote:\n>> On Mon, Feb 12, 2018 at 05:22:15PM -0800, Stefan Beller wrote:\n>>> This is a real take on the first part of the recent RFC[1].\n>>>\n>>> ...\n>>>\n>>> Duy suggested that we shall not use the repository blindly, but\n>>> should carefully examine whether to pass on an object store or the\n>>> refstore or such[4], which I agree with if it makes sense. This\n>>> series unfortunately has an issue with that as I would not want to\n>>> pass down the `ignore_env` flag separately from the object store, so\n>>> I made all functions that only take the object store to have the raw\n>>> object store as the first parameter, and others using the full\n>>> repository.\n>>\n>> Second proposal :) How about you store ignore_env in raw_object_store?\n>> This would not be the first time an object has some configuration\n>> passed in at construction time. And it has a \"constructor\" now,\n>> raw_object_store_init() (I probably should merge _setup in it too)\n>\n> A bit more on this configuration parameters. Down the road I think we\n> need something like this anyway to delete global config vars like\n> packed_git_window_size, delta_base_cache_limit... Either all these\n> end up in raw_object_store, or raw_object_store holds a link to\n> \"struct config_set\".\n\nThat makes sense long term.\n\n> The ignore_env specifically though looks to me like a stop gap\n> solution until everything goes through repo_init() first. At that\n> point we don't have to delay getenv() anymore. We can getenv() all at\n> repo_init() then pass them in raw_object_store and ignore_env should\n> be gone. So sticking it inside raw_object_store _temporarily_ does not\n> sound so bad.\n\nOh, that is an interesting perspective. Here is how I arrived at the opposite\nconclusion initially: Searching for 'ignore_env' shows that we care about it\nas well for the index and graft paths, which are not the object store, hence\nit would be better kept in the repository. (The alternative would be to\nduplicate it into the raw object store, but I do not like data duplication)\n\nBut maybe it is better to duplicate this one bit instead of passing through\na larger scoped object.\n\nI'll rework the patches.\n\nThanks!\nStefan\n"},{"id":"339218","messageId":"20180213185120.GA108526@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-3-sbeller@google.com","subject":"Re: [PATCH 02/26] object-store: move alt_odb_list and alt_odb_tail to object store","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-13T18:51:20Z","receivedAt":"2018-02-13T18:51:29Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/12, Stefan Beller wrote:\n> In a process with multiple repositories open, alternates should be\n> associated to a single repository and not shared globally. Move\n> alt_odb_list and alt_odb_tail into the_repository and adjust callers\n> to reflect this.\n> \n> Now that the alternative object data base is per repository, we're\n> leaking its memory upon freeing a repository. The next patch plugs\n> this hole.\n> \n> No functional change intended.\n> \n> Signed-off-by: Stefan Beller \n> Signed-off-by: Jonathan Nieder \n> ---\n> builtin/fsck.c | 4 +++-\n> cache.h | 4 ++--\n> object-store.h | 7 ++++++-\n> packfile.c | 3 ++-\n> sha1_file.c | 25 ++++++++++++-------------\n> sha1_name.c | 3 ++-\n> 6 files changed, 27 insertions(+), 19 deletions(-)\n> \n> diff --git a/builtin/fsck.c b/builtin/fsck.c\n> index 04846d46f9..1048255da1 100644\n> --- a/builtin/fsck.c\n> +++ b/builtin/fsck.c\n> @@ -1,5 +1,6 @@\n> #include \"builtin.h\"\n> #include \"cache.h\"\n> +#include \"repository.h\"\n> #include \"config.h\"\n> #include \"commit.h\"\n> #include \"tree.h\"\n> @@ -694,7 +695,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n> \t\tfsck_object_dir(get_object_directory());\n> \n> \t\tprepare_alt_odb();\n> -\t\tfor (alt = alt_odb_list; alt; alt = alt->next)\n> +\t\tfor (alt = the_repository->objects.alt_odb_list;\n> +\t\t\t\talt; alt = alt->next)\n> \t\t\tfsck_object_dir(alt->path);\n> \n> \t\tif (check_full) {\n> diff --git a/cache.h b/cache.h\n> index d8b975a571..918c2f15b4 100644\n> --- a/cache.h\n> +++ b/cache.h\n> @@ -1573,7 +1573,7 @@ extern int has_dirs_only_path(const char *name, int len, int prefix_len);\n> extern void schedule_dir_for_removal(const char *name, int len);\n> extern void remove_scheduled_dirs(void);\n> \n> -extern struct alternate_object_database {\n> +struct alternate_object_database {\n> \tstruct alternate_object_database *next;\n> \n> \t/* see alt_scratch_buf() */\n> @@ -1591,7 +1591,7 @@ extern struct alternate_object_database {\n> \tstruct oid_array loose_objects_cache;\n> \n> \tchar path[FLEX_ARRAY];\n> -} *alt_odb_list;\n> +};\n> extern void prepare_alt_odb(void);\n> extern char *compute_alternate_path(const char *path, struct strbuf *err);\n> typedef int alt_odb_fn(struct alternate_object_database *, void *);\n> diff --git a/object-store.h b/object-store.h\n> index cf35760ceb..44b8f84753 100644\n> --- a/object-store.h\n> +++ b/object-store.h\n> @@ -1,14 +1,19 @@\n> #ifndef OBJECT_STORE_H\n> #define OBJECT_STORE_H\n> \n> +#include \"cache.h\"\n> +\n\nMaybe we would want to move the definition of the alternate store to\nthis header file? That way we don't need to include cache.h here.\n\n> struct raw_object_store {\n> \t/*\n> \t * Path to the repository's object store.\n> \t * Cannot be NULL after initialization.\n> \t */\n> \tchar *objectdir;\n> +\n> +\tstruct alternate_object_database *alt_odb_list;\n> +\tstruct alternate_object_database **alt_odb_tail;\n> };\n> -#define RAW_OBJECT_STORE_INIT { NULL }\n> +#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }\n> \n> void raw_object_store_clear(struct raw_object_store *o);\n> \n> diff --git a/packfile.c b/packfile.c\n> index 4a5fe7ab18..d61076faaf 100644\n> --- a/packfile.c\n> +++ b/packfile.c\n> @@ -1,6 +1,7 @@\n> #include \"cache.h\"\n> #include \"mru.h\"\n> #include \"pack.h\"\n> +#include \"repository.h\"\n> #include \"dir.h\"\n> #include \"mergesort.h\"\n> #include \"packfile.h\"\n> @@ -880,7 +881,7 @@ void prepare_packed_git(void)\n> \t\treturn;\n> \tprepare_packed_git_one(get_object_directory(), 1);\n> \tprepare_alt_odb();\n> -\tfor (alt = alt_odb_list; alt; alt = alt->next)\n> +\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)\n> \t\tprepare_packed_git_one(alt->path, 0);\n> \trearrange_packed_git();\n> \tprepare_packed_git_mru();\n> diff --git a/sha1_file.c b/sha1_file.c\n> index 3da70ac650..2826d5d6ed 100644\n> --- a/sha1_file.c\n> +++ b/sha1_file.c\n> @@ -22,6 +22,7 @@\n> #include \"pack-revindex.h\"\n> #include \"sha1-lookup.h\"\n> #include \"bulk-checkin.h\"\n> +#include \"repository.h\"\n> #include \"streaming.h\"\n> #include \"dir.h\"\n> #include \"mru.h\"\n> @@ -346,9 +347,6 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,\n> \treturn buf->buf;\n> }\n> \n> -struct alternate_object_database *alt_odb_list;\n> -static struct alternate_object_database **alt_odb_tail;\n> -\n> /*\n> * Return non-zero iff the path is usable as an alternate object database.\n> */\n> @@ -368,7 +366,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)\n> \t * Prevent the common mistake of listing the same\n> \t * thing twice, or object directory itself.\n> \t */\n> -\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n> +\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n> \t\tif (!fspathcmp(path->buf, alt->path))\n> \t\t\treturn 0;\n> \t}\n> @@ -428,8 +426,8 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,\n> \tent = alloc_alt_odb(pathbuf.buf);\n> \n> \t/* add the alternate entry */\n> -\t*alt_odb_tail = ent;\n> -\talt_odb_tail = &(ent->next);\n> +\t*the_repository->objects.alt_odb_tail = ent;\n> +\tthe_repository->objects.alt_odb_tail = &(ent->next);\n> \tent->next = NULL;\n> \n> \t/* recursively add alternates */\n> @@ -563,7 +561,7 @@ void add_to_alternates_file(const char *reference)\n> \t\tfprintf_or_die(out, \"%s\\n\", reference);\n> \t\tif (commit_lock_file(&lock))\n> \t\t\tdie_errno(\"unable to move new alternates file into place\");\n> -\t\tif (alt_odb_tail)\n> +\t\tif (the_repository->objects.alt_odb_tail)\n> \t\t\tlink_alt_odb_entries(reference, '\\n', NULL, 0);\n> \t}\n> \tfree(alts);\n> @@ -661,7 +659,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n> \tint r = 0;\n> \n> \tprepare_alt_odb();\n> -\tfor (ent = alt_odb_list; ent; ent = ent->next) {\n> +\tfor (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {\n> \t\tr = fn(ent, cb);\n> \t\tif (r)\n> \t\t\tbreak;\n> @@ -673,12 +671,13 @@ void prepare_alt_odb(void)\n> {\n> \tconst char *alt;\n> \n> -\tif (alt_odb_tail)\n> +\tif (the_repository->objects.alt_odb_tail)\n> \t\treturn;\n> \n> \talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n> \n> -\talt_odb_tail = &alt_odb_list;\n> +\tthe_repository->objects.alt_odb_tail =\n> +\t\t\t&the_repository->objects.alt_odb_list;\n> \tlink_alt_odb_entries(alt, PATH_SEP, NULL, 0);\n> \n> \tread_info_alternates(get_object_directory(), 0);\n> @@ -717,7 +716,7 @@ static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)\n> {\n> \tstruct alternate_object_database *alt;\n> \tprepare_alt_odb();\n> -\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n> +\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n> \t\tconst char *path = alt_sha1_path(alt, sha1);\n> \t\tif (check_and_freshen_file(path, freshen))\n> \t\t\treturn 1;\n> @@ -873,7 +872,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n> \n> \tprepare_alt_odb();\n> \terrno = ENOENT;\n> -\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n> +\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n> \t\t*path = alt_sha1_path(alt, sha1);\n> \t\tif (!lstat(*path, st))\n> \t\t\treturn 0;\n> @@ -899,7 +898,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n> \tmost_interesting_errno = errno;\n> \n> \tprepare_alt_odb();\n> -\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n> +\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n> \t\t*path = alt_sha1_path(alt, sha1);\n> \t\tfd = git_open(*path);\n> \t\tif (fd >= 0)\n> diff --git a/sha1_name.c b/sha1_name.c\n> index 611c7d24dd..957ce25680 100644\n> --- a/sha1_name.c\n> +++ b/sha1_name.c\n> @@ -10,6 +10,7 @@\n> #include \"dir.h\"\n> #include \"sha1-array.h\"\n> #include \"packfile.h\"\n> +#include \"repository.h\"\n> \n> static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);\n> \n> @@ -104,7 +105,7 @@ static void find_short_object_filename(struct disambiguate_state *ds)\n> \t\t */\n> \t\tfakeent = alloc_alt_odb(get_object_directory());\n> \t}\n> -\tfakeent->next = alt_odb_list;\n> +\tfakeent->next = the_repository->objects.alt_odb_list;\n> \n> \tfor (alt = fakeent; alt && !ds->ambiguous; alt = alt->next) {\n> \t\tint pos;\n> -- \n> 2.16.1.73.ga2c3e9663f.dirty\n> \n\n-- \nBrandon Williams\n"},{"id":"339221","messageId":"xmqq7ergra2y.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"CAGZ79kbmAkN5EpW1hzW-FFdR74c-qXJc5aKZX0ff9bU0gxMRqQ@mail.gmail.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-13T18:57:57Z","receivedAt":"2018-02-13T18:58:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> Oh, that is an interesting perspective. Here is how I arrived at the opposite\n> conclusion initially: Searching for 'ignore_env' shows that we care about it\n> as well for the index and graft paths, which are not the object store, hence\n> it would be better kept in the repository. (The alternative would be to\n> duplicate it into the raw object store, but I do not like data duplication)\n>\n> But maybe it is better to duplicate this one bit instead of passing through\n> a larger scoped object.\n\nIf a larger scoped repository refers to a smaller scoped\nobject-store, is there still a need to duplicate that bit, instead\nof referring to the bit the smaller scoped one has when asking about\nthe bit in the larger scoped one?\n\nI am not sure if these \"do not look at environment variables\" is an\nattribute of these entities---it sounds more like an attribute for\neach invocation of an operation, i.e. \"I want to learn where the\nindex is but would ignore GIT_INDEX environment for this particular\nquery.\" and \"What's the value of this ref? Please honor the\ncommon-dir environment during this query\".\n\nSo from that point of view, it may not matter where the \"bit\" lives,\namong repository, object-store, or ref-store.\n\n"},{"id":"339222","messageId":"CAGZ79kZOq+_NOOabBUtPJ_zKNb_vdais70F6VzVFoSn5B5A4fQ@mail.gmail.com","threadId":"47823","inReplyTo":"xmqq7ergra2y.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T19:23:04Z","receivedAt":"2018-02-13T19:23:11Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Feb 13, 2018 at 10:57 AM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> Oh, that is an interesting perspective. Here is how I arrived at the opposite\n>> conclusion initially: Searching for 'ignore_env' shows that we care about it\n>> as well for the index and graft paths, which are not the object store, hence\n>> it would be better kept in the repository. (The alternative would be to\n>> duplicate it into the raw object store, but I do not like data duplication)\n>>\n>> But maybe it is better to duplicate this one bit instead of passing through\n>> a larger scoped object.\n>\n> If a larger scoped repository refers to a smaller scoped\n> object-store, is there still a need to duplicate that bit, instead\n> of referring to the bit the smaller scoped one has when asking about\n> the bit in the larger scoped one?\n\nNo (in theory). But in practice it may be worthwhile:\n\n \"What's the value of this ref?\"\n\n \"Oh let me check the ignore_env bit that happens\n to live in the object store first.\"\n\nwould be super confusing to me.\n\n> I am not sure if these \"do not look at environment variables\" is an\n> attribute of these entities---it sounds more like an attribute for\n> each invocation of an operation, i.e. \"I want to learn where the\n> index is but would ignore GIT_INDEX environment for this particular\n> query.\" and \"What's the value of this ref? Please honor the\n> common-dir environment during this query\".\n\nThat sounds like we want to have a configset struct eventually.\n\nFor now the ignore_env bit lives in the repository, as that helps\nwhen working with submodules, when reading its comments.\nUnfortunately 359efeffc1 (repository: introduce the repository\nobject, 2017-06-22) did not reason about the existence of the ignore_env\nflag in its commit message.\n\n> So from that point of view, it may not matter where the \"bit\" lives,\n> among repository, object-store, or ref-store.\n\nIt matters on the scale of confusing the developer?\n\nStefan\n"},{"id":"339223","messageId":"20180213192621.GC42272@aiede.svl.corp.google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2018-02-13T19:26:21Z","receivedAt":"2018-02-13T19:26:31Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nStefan Beller wrote:\n\n> This is a real take on the first part of the recent RFC[1].\n>\n> Jonathan Tan suggested[2] that \"sha1_loose_object_info to handle arbitrary repositories\"\n> might be a good breaking point for a first part at that RFC at patch 38.\n> This series is smaller and contains only 26 patches as the patches in the big\n> RFC were slightly out of order.\n\nThanks. This looks like a nice reviewable series, so I'm happy to see\nit broken out.\n\n[...]\n> Comments in the early range of that RFC were on 003 where Junio pointed out\n> that the coccinelle patch ought to be not in contrib/coccinelle, so I put it\n> in a sub directory there, as 'make coccicheck' doesn't traverse subdirs.\n\nCan you say a little more about this? Was the problem that the\nsemantic patch wasn't idempotent, that it was too slow to run, or\nsomething else?\n\nIf we're including the semantic patch for reference but never running\nit, I think I'd prefer it to go in the commit message. But if it's\nuseful to run then we should make it idempotent so it can go in\ncontrib/coccinelle.\n\n[...]\n> Duy suggested that we shall not use the repository blindly, but should carefully\n> examine whether to pass on an object store or the refstore or such[4], which\n> I agree with if it makes sense. This series unfortunately has an issue with that\n> as I would not want to pass down the `ignore_env` flag separately from the object\n> store, so I made all functions that only take the object store to have the raw\n> object store as the first parameter, and others using the full repository.\n\nI think I want to push back on this a little.\n\nThe advantage of a function taking e.g. an object_store as an argument\ninstead of a repository is that it increases its flexibility, since it\nallows callers that do not have access to a repository to call it. The\ndisadvantage is also that it increases the flexibility without any\ncallers benefitting from that:\n\n 1. It ties us to assumptions from today. If e.g. an object access in\n the future starts relying on some other information from the\n repository (e.g. its config) then we'd have to either add a\n back-pointer from the object store to its repository or add\n additional arguments for that additional data at that point.\n\n If all callers already have a repository, it is simpler to pass\n that repository as context so that we have the flexibility to make\n more use of it later.\n\n 2. It complicates the caller. Instead of consistently passing the\n same repository argument as context to functions that access that\n repository, the caller would have to pull out relevant fields like\n the object store from it.\n\n 3. It prevents us from making opportunistic use of other information\n from the repository, such as its name for use in error messages.\n\nIn lower-level funcitons that need to be usable by callers without a\nrepository (e.g. to find packfiles in an alternate) it makes sense to\nnot pass a repository, but without such a use case in mind I don't\nthink it needs to be a general goal.\n\nTo put it another way, most callers do not *care* whether they are\nworking with a repository's object store, ref database, or some other\naspect of the repository. They just know they want to e.g. read an\nobject from this repository.\n\nIt's similar to how FILE * works: some operations rely on the buffer\nthe FILE * manages and some other operations only rely on the\nunderlying file descriptor, but using the FILE * consistently provides\na clean abstraction that generally makes life easier.\n\n> Eric Sunshine brought up memory leaks with the RFC, and I would think to\n> have plugged all holes.\n\nYay, thank you!\n\nI'll try to find time to look at the patches in detail soon, but no\npromises (i.e. if someone else reviews them first, then even better\n;-)).\n\nSincerely,\nJonathan\n"},{"id":"339224","messageId":"20180213193337.GB108526@google.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-13T19:33:37Z","receivedAt":"2018-02-13T19:33:47Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/12, Stefan Beller wrote:\n> This is a real take on the first part of the recent RFC[1].\n> \n> Jonathan Tan suggested[2] that \"sha1_loose_object_info to handle arbitrary repositories\"\n> might be a good breaking point for a first part at that RFC at patch 38.\n> This series is smaller and contains only 26 patches as the patches in the big\n> RFC were slightly out of order.\n> \n> I developed this series partly by writing patches, but mostly by cherrypicking\n> from that RFC on top of current master. I noticed no external conflicts apart\n> from one addition to the repositories _INIT macro, which was easy to resolve.\n> \n> Comments in the early range of that RFC were on 003 where Junio pointed out\n> that the coccinelle patch ought to be not in contrib/coccinelle, so I put it\n> in a sub directory there, as 'make coccicheck' doesn't traverse subdirs.\n> \n> brian had a questoin on patch 25 in the RFC, but that seemed to resolve itself\n> without any suggestion to include into this series[3].\n> \n> Duy suggested that we shall not use the repository blindly, but should carefully\n> examine whether to pass on an object store or the refstore or such[4], which \n> I agree with if it makes sense. This series unfortunately has an issue with that\n> as I would not want to pass down the `ignore_env` flag separately from the object\n> store, so I made all functions that only take the object store to have the raw\n> object store as the first parameter, and others using the full repository.\n> \n> Eric Sunshine brought up memory leaks with the RFC, and I would think to\n> have plugged all holes.\n\nI've looked through the patches and I think they look good. At the end\nof the series all the #define tricks have been eliminated so we don't\nhave to worry about them possibly being left and forgotten :)\n\n\nThanks for getting the ball rolling on this.\n\n> \n> [1] https://public-inbox.org/git/20180205235508.216277-1-sbeller@google.com/\n> [2] https://public-inbox.org/git/20180207143300.ce1c39ca07f6a0d64fe0e7ca@google.com/\n> [3] https://public-inbox.org/git/20180206011940.GD7904@genre.crustytoothpaste.net/\n> [4] https://public-inbox.org/git/CACsJy8CGgekpX4cZkyyTSPrj87uQVKZSOL7fyT__P2dh_1LmVQ@mail.gmail.com/\n> \n> Thanks,\n> Stefan\n> \n> Jonathan Nieder (8):\n> pack: move prepare_packed_git_run_once to object store\n> pack: move approximate object count to object store\n> sha1_file: add repository argument to sha1_file_name\n> sha1_file: add repository argument to map_sha1_file\n> sha1_file: allow stat_sha1_file to handle arbitrary repositories\n> sha1_file: allow open_sha1_file to handle arbitrary repositories\n> sha1_file: allow map_sha1_file_1 to handle arbitrary repositories\n> sha1_file: allow sha1_loose_object_info to handle arbitrary\n> repositories\n> \n> Stefan Beller (18):\n> repository: introduce raw object store field\n> object-store: move alt_odb_list and alt_odb_tail to object store\n> object-store: free alt_odb_list\n> object-store: move packed_git and packed_git_mru to object store\n> object-store: close all packs upon clearing the object store\n> sha1_file: add raw_object_store argument to alt_odb_usable\n> sha1_file: add repository argument to link_alt_odb_entry\n> sha1_file: add repository argument to read_info_alternates\n> sha1_file: add repository argument to link_alt_odb_entries\n> sha1_file: add repository argument to prepare_alt_odb\n> sha1_file: allow link_alt_odb_entries to handle arbitrary repositories\n> sha1_file: allow prepare_alt_odb to handle arbitrary repositories\n> sha1_file: add repository argument to stat_sha1_file\n> sha1_file: add repository argument to open_sha1_file\n> sha1_file: add repository argument to map_sha1_file_1\n> sha1_file: add repository argument to sha1_loose_object_info\n> sha1_file: allow sha1_file_name to handle arbitrary repositories\n> sha1_file: allow map_sha1_file to handle arbitrary repositories\n> \n> builtin/am.c | 2 +-\n> builtin/clone.c | 2 +-\n> builtin/count-objects.c | 6 +-\n> builtin/fetch.c | 2 +-\n> builtin/fsck.c | 13 ++-\n> builtin/gc.c | 4 +-\n> builtin/grep.c | 2 +-\n> builtin/index-pack.c | 1 +\n> builtin/merge.c | 2 +-\n> builtin/pack-objects.c | 21 ++--\n> builtin/pack-redundant.c | 6 +-\n> builtin/receive-pack.c | 3 +-\n> cache.h | 46 ++------\n> contrib/coccinelle/refactoring/packed_git.cocci | 7 ++\n> environment.c | 5 +-\n> fast-import.c | 6 +-\n> http-backend.c | 6 +-\n> http-push.c | 1 +\n> http-walker.c | 4 +-\n> http.c | 6 +-\n> mru.h | 1 +\n> object-store.h | 75 +++++++++++++\n> object.c | 26 +++++\n> pack-bitmap.c | 4 +-\n> pack-check.c | 1 +\n> pack-revindex.c | 1 +\n> packfile.c | 64 +++++------\n> packfile.h | 2 +-\n> path.c | 2 +-\n> reachable.c | 1 +\n> repository.c | 17 ++-\n> repository.h | 7 +-\n> server-info.c | 6 +-\n> sha1_file.c | 135 +++++++++++++-----------\n> sha1_name.c | 11 +-\n> streaming.c | 5 +-\n> 36 files changed, 321 insertions(+), 182 deletions(-)\n> create mode 100644 contrib/coccinelle/refactoring/packed_git.cocci\n> create mode 100644 object-store.h\n> \n> -- \n> 2.16.1.73.ga2c3e9663f.dirty\n> \n\n-- \nBrandon Williams\n"},{"id":"339225","messageId":"xmqqzi4cptsa.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"CAGZ79kZOq+_NOOabBUtPJ_zKNb_vdais70F6VzVFoSn5B5A4fQ@mail.gmail.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-13T19:35:17Z","receivedAt":"2018-02-13T19:35:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> For now the ignore_env bit lives in the repository, as that helps\n> when working with submodules, when reading its comments.\n> Unfortunately 359efeffc1 (repository: introduce the repository\n> object, 2017-06-22) did not reason about the existence of the ignore_env\n> flag in its commit message.\n>\n>> So from that point of view, it may not matter where the \"bit\" lives,\n>> among repository, object-store, or ref-store.\n>\n> It matters on the scale of confusing the developer?\n\nWhat I meant is that from the point of view, having the bit in the\ndata (not on the invocation) is confusing no matter which data\nstructure holds it--they are equally bad.\n"},{"id":"339226","messageId":"CAGZ79kY4bFHAYeVn_8jBKp+kNzCs+juaa0Lp=Tx=SZK_HtDQHg@mail.gmail.com","threadId":"47823","inReplyTo":"xmqqzi4cptsa.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T19:43:04Z","receivedAt":"2018-02-13T19:43:10Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Feb 13, 2018 at 11:35 AM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> For now the ignore_env bit lives in the repository, as that helps\n>> when working with submodules, when reading its comments.\n>> Unfortunately 359efeffc1 (repository: introduce the repository\n>> object, 2017-06-22) did not reason about the existence of the ignore_env\n>> flag in its commit message.\n>>\n>>> So from that point of view, it may not matter where the \"bit\" lives,\n>>> among repository, object-store, or ref-store.\n>>\n>> It matters on the scale of confusing the developer?\n>\n> What I meant is that from the point of view, having the bit in the\n> data (not on the invocation) is confusing no matter which data\n> structure holds it--they are equally bad.\n\nRight.\n\nWhich is why I'd strongly consider having it only in the repository\nobject as that is the largest-scoped thing we'd want. e.g. submodules\nshould care about environment variables differently:\n\n GIT_WORK_TREE=~/mysuperproject git checkout \\\n --recurse-submodules master\n\n So with such a command in mind, the environment variable would\nonly apply to the superproject and the nested submodules should\nignore the env, but compute their paths off the superproject, i.e.\nthe superproject repository, not its object store?\n"},{"id":"339228","messageId":"CAGZ79kZQGbf-0n6vfnF9=tneaKnt_X6odc5OUJbD9X5z8JTcoA@mail.gmail.com","threadId":"47823","inReplyTo":"20180213185120.GA108526@google.com","subject":"Re: [PATCH 02/26] object-store: move alt_odb_list and alt_odb_tail to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-13T19:46:22Z","receivedAt":"2018-02-13T19:46:32Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Feb 13, 2018 at 10:51 AM, Brandon Williams wrote:\n\n>> +#include \"cache.h\"\n>> +\n>\n> Maybe we would want to move the definition of the alternate store to\n> this header file? That way we don't need to include cache.h here.\n>\n\nSounds good. I'll take a look at that.\n\nStefan\n"},{"id":"339247","messageId":"xmqqmv0cpg3n.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"CAGZ79kY4bFHAYeVn_8jBKp+kNzCs+juaa0Lp=Tx=SZK_HtDQHg@mail.gmail.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-14T00:30:52Z","receivedAt":"2018-02-14T00:31:00Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> Which is why I'd strongly consider having it only in the repository\n> object as that is the largest-scoped thing we'd want. e.g. submodules\n> should care about environment variables differently:\n>\n> GIT_WORK_TREE=~/mysuperproject git checkout \\\n> --recurse-submodules master\n>\n> So with such a command in mind, the environment variable would\n> only apply to the superproject and the nested submodules should\n> ignore the env, but compute their paths off the superproject, i.e.\n> the superproject repository, not its object store?\n\nIn the longer term endgame state, what is in the repository object\ncannot just be a simple \"do we honor environment variable\" bit\nanymore. \n\nIt will be more like \"we may (or may not) look at environment when\nwe create a repository object\", i.e. a bit passed to repo_init()\nconstructor, and from then on, the repository object knows where the\nobject store and its alternates are, where the top of the working\ntree is, where the repository (i.e. the directory that has \"refs/\"\nin it) is, what \"worktree\" of the repository we are talking about by\nitself. There is no need for a bit \"do we or do we not look at\nenvironment?\" that needs to be consulted at runtime, which is quite\na round-about thing.\n\nIn your example, the repository object that represents the\nsuperproject will pay attention to GIT_WORK_TREE environment when it\nis consturcted, and repository objects dynamically constructed while\nthe command \"recurse-submodules\" through them will be constructed\nwith their \"where is the top of my working tree?\" set appropriately.\nThey won't be storing \"when figuring out where my working tree is,\ndo not pay attention to GIT_WORK_TREE environment variable\" bit.\n\n"},{"id":"339248","messageId":"CACsJy8DmXrO9bxv10DsDpSWxm2BiZwwfhso4=ZRQs_xhrpAgfg@mail.gmail.com","threadId":"47823","inReplyTo":"20180213012241.187007-15-sbeller@google.com","subject":"Re: [PATCH 14/26] sha1_file: allow prepare_alt_odb to handle arbitrary repositories","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-14T00:37:53Z","receivedAt":"2018-02-14T00:38:29Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Feb 13, 2018 at 8:22 AM, Stefan Beller wrote:\n> Signed-off-by: Stefan Beller \n> ---\n> object-store.h | 3 +--\n> sha1_file.c | 21 +++++++++++----------\n> 2 files changed, 12 insertions(+), 12 deletions(-)\n>\n> diff --git a/object-store.h b/object-store.h\n> index d96a16edd1..add1d4e27c 100644\n> --- a/object-store.h\n> +++ b/object-store.h\n> @@ -61,7 +61,6 @@ struct packed_git {\n> char pack_name[FLEX_ARRAY]; /* more */\n> };\n>\n> -#define prepare_alt_odb(r) prepare_alt_odb_##r()\n> -extern void prepare_alt_odb_the_repository(void);\n> +void prepare_alt_odb(struct repository *r);\n>\n> #endif /* OBJECT_STORE_H */\n> diff --git a/sha1_file.c b/sha1_file.c\n> index d18ce3aeba..f046d560f8 100644\n> --- a/sha1_file.c\n> +++ b/sha1_file.c\n> @@ -677,21 +677,22 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n> return r;\n> }\n>\n> -void prepare_alt_odb_the_repository(void)\n> +void prepare_alt_odb(struct repository *r)\n> {\n> - const char *alt;\n> -\n> - if (the_repository->objects.alt_odb_tail)\n> + if (r->objects.alt_odb_tail)\n> return;\n>\n> - alt = getenv(ALTERNATE_DB_ENVIRONMENT);\n> + r->objects.alt_odb_tail = &r->objects.alt_odb_list;\n> +\n> + if (!r->ignore_env) {\n> + const char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);\n\nIf one day the majority of git moves to use 'struct repository', then\nALTERNATE_DB_ENVIRONMENT is always ignored because ignore_env is\nalways true. I think if you ignore_env, then you still need to get\nthis \"alt\" from 'struct raw_object_store' (or 'struct repository').\n\nSince you get lots of getenv() in repo_setup_env(), I think this\ngetenv(ALTERNATE_DB_ENVIRONMENT) belongs there too. Then here, if\nignore_env is true, you read r->objects.alt or something instead of\ndoing getenv().\n\nI really want to kill this getenv() in this code, which is basically\ndelayed initialization because we haven't done proper init on\nthe_repo. I realize that it cannot be done earlier, when\nprepare_alt_odb() does not even have a 'struct repository *' to work\nwith. Would it be ok if I contributed a patch on top of your series to\nbasically do repo_init(&the_repo) for all builtin/external commands\n(and fix all the bugs that come with it)? Then we would not need\nignore_env here anymore.\n\n> + if (!alt)\n> + alt = \"\";\n>\n> - the_repository->objects.alt_odb_tail =\n> - &the_repository->objects.alt_odb_list;\n> - link_alt_odb_entries(the_repository, alt,\n> - PATH_SEP, NULL, 0);\n> + link_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);\n> + }\n>\n> - read_info_alternates(the_repository, get_object_directory(), 0);\n> + read_info_alternates(r, r->objects.objectdir, 0);\n> }\n>\n> /* Returns 1 if we have successfully freshened the file, 0 otherwise. */\n> --\n> 2.16.1.73.ga2c3e9663f.dirty\n>\n\n\n\n-- \nDuy\n"},{"id":"339249","messageId":"CACsJy8CPKESE8atc_eWdNVknQYp9T6ebwKwCdzLHyaFKH2BnZA@mail.gmail.com","threadId":"47823","inReplyTo":"20180213192621.GC42272@aiede.svl.corp.google.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-14T00:57:06Z","receivedAt":"2018-02-14T00:57:42Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Feb 14, 2018 at 2:26 AM, Jonathan Nieder wrote:\n>> Duy suggested that we shall not use the repository blindly, but should carefully\n>> examine whether to pass on an object store or the refstore or such[4], which\n>> I agree with if it makes sense. This series unfortunately has an issue with that\n>> as I would not want to pass down the `ignore_env` flag separately from the object\n>> store, so I made all functions that only take the object store to have the raw\n>> object store as the first parameter, and others using the full repository.\n>\n> I think I want to push back on this a little.\n>\n> The advantage of a function taking e.g. an object_store as an argument\n> instead of a repository is that it increases its flexibility, since it\n> allows callers that do not have access to a repository to call it. The\n> disadvantage is also that it increases the flexibility without any\n> callers benefitting from that:\n>\n> 1. It ties us to assumptions from today. If e.g. an object access in\n> the future starts relying on some other information from the\n> repository (e.g. its config) then we'd have to either add a\n> back-pointer from the object store to its repository or add\n> additional arguments for that additional data at that point.\n>\n> If all callers already have a repository, it is simpler to pass\n> that repository as context so that we have the flexibility to make\n> more use of it later.\n\nIt's essentially putting all global variables in the same place again.\nOnly this time it's not global namespace, but \"struct repository\".\nIt's no worse than the current state though.\n\n>\n> 2. It complicates the caller. Instead of consistently passing the\n> same repository argument as context to functions that access that\n> repository, the caller would have to pull out relevant fields like\n> the object store from it.\n\nWell, I see that as a good point :)\n\n>\n> 3. It prevents us from making opportunistic use of other information\n> from the repository, such as its name for use in error messages.\n\nIt does not exactly prevent us. It's just more effort to pass this\naround. The repository name for example, there's no reason we can't\nhave object store name (which could be initialized the same as repo\nname).\n\n> In lower-level funcitons that need to be usable by callers without a\n> repository (e.g. to find packfiles in an alternate) it makes sense to\n> not pass a repository, but without such a use case in mind\n\nI do agree with your benefit argument. But I'd like to point out\nthough that having all input to object store visible from something\nlike \"struct raw_object_store\" makes it easier to reason about the\ncode. I know how object store works, but occasionally I'm still\nsurprised when it getenv (or read $GIT_DIR/index, but not in object\nstore code) behind the scene. Imagine how hard it is for newcomers.\n\nI would count that as benefit, even though it's not a use case per se.\nAnother potential benefit is writing unit tests will be much easier\n(you can configure object store through struct repository too, but\nsetting one piece here, one piece there to control object store\nbehavior is not a nice experience). It's a nice thing to have, but not\na deciding factor.\n\n> I don't think it needs to be a general goal.\n\nMy stand is a bit more aggressive. We should try to achieve better\nabstraction if possible. But if it makes Stefan's life hell, it's not\nworth doing. Converting to 'struct repository' is already a step\nforward. Actually if it discourages him from finishing this work, it's\nalready not worth doing.\n-- \nDuy\n"},{"id":"339300","messageId":"20180214180814.GA139458@google.com","threadId":"47823","inReplyTo":"CACsJy8DmXrO9bxv10DsDpSWxm2BiZwwfhso4=ZRQs_xhrpAgfg@mail.gmail.com","subject":"Re: [PATCH 14/26] sha1_file: allow prepare_alt_odb to handle arbitrary repositories","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-14T18:08:14Z","receivedAt":"2018-02-14T18:08:22Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/14, Duy Nguyen wrote:\n> On Tue, Feb 13, 2018 at 8:22 AM, Stefan Beller wrote:\n> > Signed-off-by: Stefan Beller \n> > ---\n> > object-store.h | 3 +--\n> > sha1_file.c | 21 +++++++++++----------\n> > 2 files changed, 12 insertions(+), 12 deletions(-)\n> >\n> > diff --git a/object-store.h b/object-store.h\n> > index d96a16edd1..add1d4e27c 100644\n> > --- a/object-store.h\n> > +++ b/object-store.h\n> > @@ -61,7 +61,6 @@ struct packed_git {\n> > char pack_name[FLEX_ARRAY]; /* more */\n> > };\n> >\n> > -#define prepare_alt_odb(r) prepare_alt_odb_##r()\n> > -extern void prepare_alt_odb_the_repository(void);\n> > +void prepare_alt_odb(struct repository *r);\n> >\n> > #endif /* OBJECT_STORE_H */\n> > diff --git a/sha1_file.c b/sha1_file.c\n> > index d18ce3aeba..f046d560f8 100644\n> > --- a/sha1_file.c\n> > +++ b/sha1_file.c\n> > @@ -677,21 +677,22 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n> > return r;\n> > }\n> >\n> > -void prepare_alt_odb_the_repository(void)\n> > +void prepare_alt_odb(struct repository *r)\n> > {\n> > - const char *alt;\n> > -\n> > - if (the_repository->objects.alt_odb_tail)\n> > + if (r->objects.alt_odb_tail)\n> > return;\n> >\n> > - alt = getenv(ALTERNATE_DB_ENVIRONMENT);\n> > + r->objects.alt_odb_tail = &r->objects.alt_odb_list;\n> > +\n> > + if (!r->ignore_env) {\n> > + const char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);\n> \n> If one day the majority of git moves to use 'struct repository', then\n> ALTERNATE_DB_ENVIRONMENT is always ignored because ignore_env is\n> always true. I think if you ignore_env, then you still need to get\n> this \"alt\" from 'struct raw_object_store' (or 'struct repository').\n> \n> Since you get lots of getenv() in repo_setup_env(), I think this\n> getenv(ALTERNATE_DB_ENVIRONMENT) belongs there too. Then here, if\n> ignore_env is true, you read r->objects.alt or something instead of\n> doing getenv().\n> \n> I really want to kill this getenv() in this code, which is basically\n> delayed initialization because we haven't done proper init on\n> the_repo. I realize that it cannot be done earlier, when\n> prepare_alt_odb() does not even have a 'struct repository *' to work\n> with. Would it be ok if I contributed a patch on top of your series to\n> basically do repo_init(&the_repo) for all builtin/external commands\n> (and fix all the bugs that come with it)? Then we would not need\n> ignore_env here anymore.\n\nAt some point yes we would definitely want the setup code to fully\ninitialize a repository object (in this case the_repository). And I\nwould even like to change the function signatures of all the builtin\ncommands to take a repository object so that they don't implicitly rely\non the_repository at all.\n\nWhen I introduced struct repository I seem to remember there being a\ncouple things which were different about setup that made it difficult to\nsimply call repo_init() on the_repository. If you can fix whatever\nthose issues with setup were (I can't remember all of them) then that\nwould be great :)\n\n> \n> > + if (!alt)\n> > + alt = \"\";\n> >\n> > - the_repository->objects.alt_odb_tail =\n> > - &the_repository->objects.alt_odb_list;\n> > - link_alt_odb_entries(the_repository, alt,\n> > - PATH_SEP, NULL, 0);\n> > + link_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);\n> > + }\n> >\n> > - read_info_alternates(the_repository, get_object_directory(), 0);\n> > + read_info_alternates(r, r->objects.objectdir, 0);\n> > }\n> >\n> > /* Returns 1 if we have successfully freshened the file, 0 otherwise. */\n> > --\n> > 2.16.1.73.ga2c3e9663f.dirty\n> >\n> \n> \n> \n> -- \n> Duy\n\n-- \nBrandon Williams\n"},{"id":"339501","messageId":"xmqqlgfu7zn5.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"20180213012241.187007-1-sbeller@google.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-15T20:42:54Z","receivedAt":"2018-02-15T20:43:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> This is a real take on the first part of the recent RFC[1].\n\nFor the patches remaining in this series, The scope is about right\nand the size is more manageable. With topics already on 'master',\nthey have some interactions:\n\n - ot/mru-on-list & gs/retire-mru\n\n The packed_git MRU has been greatly simplified by using list API\n directly.\n\n - cc/sha1-file-name\n\n The function signature of sha1_file_name() has been updated.\n\nI could certainly carry evil merge resolutions going forward (these\nchanges need to be made to object-store.h that has no mechanical\nconflicts), but it may be less distracting for later readers of the\nseries if we rebase it on top of a more recent master.\n"},{"id":"339505","messageId":"CAGZ79kY5_aL0YRE5JMp4Y=J_yKA0QfoB-LzXxQH0NN8iU-q3TA@mail.gmail.com","threadId":"47823","inReplyTo":"xmqqlgfu7zn5.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 00/26] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-15T21:09:20Z","receivedAt":"2018-02-15T21:09:27Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Feb 15, 2018 at 12:42 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> This is a real take on the first part of the recent RFC[1].\n>\n> For the patches remaining in this series, The scope is about right\n> and the size is more manageable.\n\nok, thanks.\n\n> With topics already on 'master',\n> they have some interactions:\n>\n> - ot/mru-on-list & gs/retire-mru\n>\n> The packed_git MRU has been greatly simplified by using list API\n> directly.\n>\n> - cc/sha1-file-name\n>\n> The function signature of sha1_file_name() has been updated.\n>\n> I could certainly carry evil merge resolutions going forward (these\n> changes need to be made to object-store.h that has no mechanical\n> conflicts), but it may be less distracting for later readers of the\n> series if we rebase it on top of a more recent master.\n\nI was debating when to reroll this series as I want to make the\nchange that Duy proposed, moving the 'ignore_env' into the\nobject store as well.\n\nI'll rebase on top of the latest master or these topicc while at it.\n\nThanks for the heads up.\nStefan\n"},{"id":"339543","messageId":"20180216174626.24677-1-sbeller@google.com","threadId":"47823","inReplyTo":"xmqqlgfu7zn5.fsf@gitster-ct.c.googlers.com","subject":"[PATCHv2 00/16] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:10Z","receivedAt":"2018-02-16T17:46:40Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"v2:\n* duplicated the 'ignore_env' bit into the object store as well\n* the #define trick no longer works as we do not have a \"the_objectstore\" global,\n which means there is just one patch per function that is converted.\n As this follows the same structure of the previous series, I am still confident\n there is no hidden dependencies to globals outside the object store in these\n converted functions.\n* rebased on top of current master, resolving the merge conflicts.\n I think I used the list.h APIs right, but please double check.\n \nThanks,\nStefan\n\nv1:\nThis is a real take on the first part of the recent RFC[1].\n\nJonathan Tan suggested[2] that \"sha1_loose_object_info to handle arbitrary repositories\"\nmight be a good breaking point for a first part at that RFC at patch 38.\nThis series is smaller and contains only 26 patches as the patches in the big\nRFC were slightly out of order.\n\nI developed this series partly by writing patches, but mostly by cherrypicking\nfrom that RFC on top of current master. I noticed no external conflicts apart\nfrom one addition to the repositories _INIT macro, which was easy to resolve.\n\nComments in the early range of that RFC were on 003 where Junio pointed out\nthat the coccinelle patch ought to be not in contrib/coccinelle, so I put it\nin a sub directory there, as 'make coccicheck' doesn't traverse subdirs.\n\nbrian had a questoin on patch 25 in the RFC, but that seemed to resolve itself\nwithout any suggestion to include into this series[3].\n\nDuy suggested that we shall not use the repository blindly, but should carefully\nexamine whether to pass on an object store or the refstore or such[4], which\nI agree with if it makes sense. This series unfortunately has an issue with that\nas I would not want to pass down the `ignore_env` flag separately from the object\nstore, so I made all functions that only take the object store to have the raw\nobject store as the first parameter, and others using the full repository.\n\nEric Sunshine brought up memory leaks with the RFC, and I would think to\nhave plugged all holes.\n\n[1] https://public-inbox.org/git/20180205235508.216277-1-sbeller@google.com/\n[2] https://public-inbox.org/git/20180207143300.ce1c39ca07f6a0d64fe0e7ca@google.com/\n[3] https://public-inbox.org/git/20180206011940.GD7904@genre.crustytoothpaste.net/\n[4] https://public-inbox.org/git/CACsJy8CGgekpX4cZkyyTSPrj87uQVKZSOL7fyT__P2dh_1LmVQ@mail.gmail.com/\n\nThanks,\nStefan\n\nStefan Beller (16):\n repository: introduce raw object store field\n object-store: move alt_odb_list and alt_odb_tail to object store\n object-store: free alt_odb_list\n object-store: move packed_git and packed_git_mru to object store\n object-store: close all packs upon clearing the object store\n pack: move prepare_packed_git_run_once to object store\n pack: move approximate object count to object store\n sha1_file: add raw_object_store argument to alt_odb_usable\n sha1_file: allow link_alt_odb_entries to handle arbitrary object\n stores\n sha1_file: allow prepare_alt_odb to handle arbitrary object stores\n sha1_file: allow sha1_file_name to handle arbitrary object stores\n sha1_file: allow stat_sha1_file to handle arbitrary object stores\n sha1_file: allow open_sha1_file to handle arbitrary object stores\n sha1_file: allow map_sha1_file_1 to handle arbitrary object stores\n sha1_file: allow map_sha1_file to handle arbitrary object stores\n sha1_file: allow sha1_loose_object_info to handle arbitrary object\n stores\n\n builtin/am.c | 2 +-\n builtin/clone.c | 2 +-\n builtin/count-objects.c | 6 +-\n builtin/fetch.c | 2 +-\n builtin/fsck.c | 13 +++--\n builtin/gc.c | 4 +-\n builtin/grep.c | 2 +-\n builtin/index-pack.c | 1 +\n builtin/merge.c | 2 +-\n builtin/pack-objects.c | 19 +++---\n builtin/pack-redundant.c | 6 +-\n builtin/receive-pack.c | 3 +-\n cache.h | 45 ++------------\n environment.c | 5 +-\n fast-import.c | 6 +-\n http-backend.c | 6 +-\n http-push.c | 1 +\n http-walker.c | 4 +-\n http.c | 6 +-\n object-store.h | 80 +++++++++++++++++++++++++\n object.c | 27 +++++++++\n pack-bitmap.c | 4 +-\n pack-check.c | 1 +\n pack-revindex.c | 1 +\n packfile.c | 64 ++++++++++----------\n packfile.h | 2 +-\n path.c | 2 +-\n reachable.c | 1 +\n repository.c | 22 +++++--\n repository.h | 7 ++-\n server-info.c | 6 +-\n sha1_file.c | 123 +++++++++++++++++++++------------------\n sha1_name.c | 11 ++--\n streaming.c | 5 +-\n 34 files changed, 314 insertions(+), 177 deletions(-)\n create mode 100644 object-store.h\n\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339544","messageId":"20180216174626.24677-2-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 01/16] repository: introduce raw object store field","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:11Z","receivedAt":"2018-02-16T17:46:43Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The raw object store field will contain any objects needed for\naccess to objects in a given repository.\n\nThis patch introduces the raw object store and populates it with the\n`objectdir`, which used to be part of the repository struct as well as\n'ignore_env' which is duplicated from the repository. A later refactoring\n(not in this series) will hopefully get rid of that one bit, once all\nthe construction of repo objects goes through repo_init(). The reason for\nduplication now is that the refactoring done in this series allows for\nsmaller scoped objects in the functions refactored, nameny passing around\nthe object store instead of the repository object.\n\nAs the struct gains members, we'll also populate the function to clear\nthe memory for these members.\n\nIn a later we'll introduce a struct object_parser, that will complement\nthe object parsing in a repository struct: The raw object parser is the\nlayer that will provide access to raw object content, while the higher\nlevel object parser code will parse raw objects and keeps track of\nparenthood and other object relationships using 'struct object'.\nFor now only add the lower level to the repository struct.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/grep.c | 2 +-\n environment.c | 5 +++--\n object-store.h | 17 +++++++++++++++++\n object.c | 5 +++++\n path.c | 2 +-\n repository.c | 19 +++++++++++++++----\n repository.h | 7 ++++---\n 7 files changed, 46 insertions(+), 11 deletions(-)\n create mode 100644 object-store.h\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 3ca4ac80d8..0f0c195705 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -432,7 +432,7 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject,\n \t * object.\n \t */\n \tgrep_read_lock();\n-\tadd_to_alternates_memory(submodule.objectdir);\n+\tadd_to_alternates_memory(submodule.objects.objectdir);\n \tgrep_read_unlock();\n \n \tif (oid) {\ndiff --git a/environment.c b/environment.c\nindex de8431e01e..ec10b062e6 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -13,6 +13,7 @@\n #include \"refs.h\"\n #include \"fmt-merge-msg.h\"\n #include \"commit.h\"\n+#include \"object-store.h\"\n \n int trust_executable_bit = 1;\n int trust_ctime = 1;\n@@ -244,9 +245,9 @@ const char *get_git_work_tree(void)\n \n char *get_object_directory(void)\n {\n-\tif (!the_repository->objectdir)\n+\tif (!the_repository->objects.objectdir)\n \t\tBUG(\"git environment hasn't been setup\");\n-\treturn the_repository->objectdir;\n+\treturn the_repository->objects.objectdir;\n }\n \n int odb_mkstemp(struct strbuf *template, const char *pattern)\ndiff --git a/object-store.h b/object-store.h\nnew file mode 100644\nindex 0000000000..5959d990fc\n--- /dev/null\n+++ b/object-store.h\n@@ -0,0 +1,17 @@\n+#ifndef OBJECT_STORE_H\n+#define OBJECT_STORE_H\n+\n+struct raw_object_store {\n+\t/*\n+\t * Path to the repository's object store.\n+\t * Cannot be NULL after initialization.\n+\t */\n+\tchar *objectdir;\n+\n+\tunsigned ignore_env : 1;\n+};\n+#define RAW_OBJECT_STORE_INIT { NULL, 0 }\n+\n+void raw_object_store_clear(struct raw_object_store *o);\n+\n+#endif /* OBJECT_STORE_H */\ndiff --git a/object.c b/object.c\nindex 9e6f9ff20b..11d904c033 100644\n--- a/object.c\n+++ b/object.c\n@@ -445,3 +445,8 @@ void clear_commit_marks_all(unsigned int flags)\n \t\t\tobj->flags &= ~flags;\n \t}\n }\n+\n+void raw_object_store_clear(struct raw_object_store *o)\n+{\n+\tfree(o->objectdir);\n+}\ndiff --git a/path.c b/path.c\nindex da8b655730..81a42d9115 100644\n--- a/path.c\n+++ b/path.c\n@@ -382,7 +382,7 @@ static void adjust_git_path(const struct repository *repo,\n \t\tstrbuf_splice(buf, 0, buf->len,\n \t\t\t repo->index_file, strlen(repo->index_file));\n \telse if (dir_prefix(base, \"objects\"))\n-\t\treplace_dir(buf, git_dir_len + 7, repo->objectdir);\n+\t\treplace_dir(buf, git_dir_len + 7, repo->objects.objectdir);\n \telse if (git_hooks_path && dir_prefix(base, \"hooks\"))\n \t\treplace_dir(buf, git_dir_len + 5, git_hooks_path);\n \telse if (repo->different_commondir)\ndiff --git a/repository.c b/repository.c\nindex 4ffbe9bc94..3b8f1b91be 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -1,11 +1,18 @@\n #include \"cache.h\"\n #include \"repository.h\"\n+#include \"object-store.h\"\n #include \"config.h\"\n #include \"submodule-config.h\"\n \n /* The main repository */\n static struct repository the_repo = {\n-\tNULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0\n+\tNULL, NULL,\n+\tRAW_OBJECT_STORE_INIT,\n+\tNULL, NULL, NULL,\n+\tNULL, NULL, NULL,\n+\t&the_index,\n+\t&hash_algos[GIT_HASH_SHA1],\n+\t0, 0\n };\n struct repository *the_repository = &the_repo;\n \n@@ -42,9 +49,11 @@ static void repo_setup_env(struct repository *repo)\n \t\t\t\t\t\t !repo->ignore_env);\n \tfree(repo->commondir);\n \trepo->commondir = strbuf_detach(&sb, NULL);\n-\tfree(repo->objectdir);\n-\trepo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,\n+\tfree(repo->objects.objectdir);\n+\trepo->objects.objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,\n \t\t\t\t\t \"objects\", !repo->ignore_env);\n+\trepo->objects.ignore_env = repo->ignore_env;\n+\n \tfree(repo->graft_file);\n \trepo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,\n \t\t\t\t\t \"info/grafts\", !repo->ignore_env);\n@@ -209,12 +218,14 @@ void repo_clear(struct repository *repo)\n {\n \tFREE_AND_NULL(repo->gitdir);\n \tFREE_AND_NULL(repo->commondir);\n-\tFREE_AND_NULL(repo->objectdir);\n \tFREE_AND_NULL(repo->graft_file);\n \tFREE_AND_NULL(repo->index_file);\n \tFREE_AND_NULL(repo->worktree);\n \tFREE_AND_NULL(repo->submodule_prefix);\n \n+\traw_object_store_clear(&repo->objects);\n+\tmemset(&repo->objects, 0, sizeof(repo->objects));\n+\n \tif (repo->config) {\n \t\tgit_configset_clear(repo->config);\n \t\tFREE_AND_NULL(repo->config);\ndiff --git a/repository.h b/repository.h\nindex 0329e40c7f..1f8bc7a7cf 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -1,6 +1,8 @@\n #ifndef REPOSITORY_H\n #define REPOSITORY_H\n \n+#include \"object-store.h\"\n+\n struct config_set;\n struct index_state;\n struct submodule_cache;\n@@ -21,10 +23,9 @@ struct repository {\n \tchar *commondir;\n \n \t/*\n-\t * Path to the repository's object store.\n-\t * Cannot be NULL after initialization.\n+\t * Holds any information related to the object store.\n \t */\n-\tchar *objectdir;\n+\tstruct raw_object_store objects;\n \n \t/*\n \t * Path to the repository's graft file.\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339545","messageId":"20180216174626.24677-4-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 03/16] object-store: free alt_odb_list","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:13Z","receivedAt":"2018-02-16T17:46:47Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Free the memory and reset alt_odb_{list, tail} to NULL.\n\nSigned-off-by: Stefan Beller \n---\n object.c | 17 +++++++++++++++++\n 1 file changed, 17 insertions(+)\n\ndiff --git a/object.c b/object.c\nindex 11d904c033..17b1da6d6b 100644\n--- a/object.c\n+++ b/object.c\n@@ -446,7 +446,24 @@ void clear_commit_marks_all(unsigned int flags)\n \t}\n }\n \n+static void free_alt_odb(struct alternate_object_database *alt)\n+{\n+\tstrbuf_release(&alt->scratch);\n+\toid_array_clear(&alt->loose_objects_cache);\n+}\n+\n+static void free_alt_odbs(struct raw_object_store *o)\n+{\n+\twhile (o->alt_odb_list) {\n+\t\tfree_alt_odb(o->alt_odb_list);\n+\t\to->alt_odb_list = o->alt_odb_list->next;\n+\t}\n+}\n+\n void raw_object_store_clear(struct raw_object_store *o)\n {\n \tfree(o->objectdir);\n+\n+\tfree_alt_odbs(o);\n+\to->alt_odb_tail = NULL;\n }\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339546","messageId":"20180216174626.24677-3-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 02/16] object-store: move alt_odb_list and alt_odb_tail to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:12Z","receivedAt":"2018-02-16T17:46:50Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a process with multiple repositories open, alternates should be\nassociated to a single repository and not shared globally. Move\nalt_odb_list and alt_odb_tail into the_repository and adjust callers\nto reflect this.\n\nNow that the alternative object data base is per repository, we're\nleaking its memory upon freeing a repository. The next patch plugs\nthis hole.\n\nNo functional change intended.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/fsck.c | 4 +++-\n cache.h | 4 ++--\n object-store.h | 7 ++++++-\n packfile.c | 3 ++-\n sha1_file.c | 25 ++++++++++++-------------\n sha1_name.c | 3 ++-\n 6 files changed, 27 insertions(+), 19 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 7a8a679d4f..908e4f092a 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -1,5 +1,6 @@\n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n #include \"commit.h\"\n #include \"tree.h\"\n@@ -716,7 +717,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t\tfsck_object_dir(get_object_directory());\n \n \t\tprepare_alt_odb();\n-\t\tfor (alt = alt_odb_list; alt; alt = alt->next)\n+\t\tfor (alt = the_repository->objects.alt_odb_list;\n+\t\t\t\talt; alt = alt->next)\n \t\t\tfsck_object_dir(alt->path);\n \n \t\tif (check_full) {\ndiff --git a/cache.h b/cache.h\nindex 9cac7bb518..f373c30e25 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1576,7 +1576,7 @@ extern int has_dirs_only_path(const char *name, int len, int prefix_len);\n extern void schedule_dir_for_removal(const char *name, int len);\n extern void remove_scheduled_dirs(void);\n \n-extern struct alternate_object_database {\n+struct alternate_object_database {\n \tstruct alternate_object_database *next;\n \n \t/* see alt_scratch_buf() */\n@@ -1594,7 +1594,7 @@ extern struct alternate_object_database {\n \tstruct oid_array loose_objects_cache;\n \n \tchar path[FLEX_ARRAY];\n-} *alt_odb_list;\n+};\n extern void prepare_alt_odb(void);\n extern char *compute_alternate_path(const char *path, struct strbuf *err);\n typedef int alt_odb_fn(struct alternate_object_database *, void *);\ndiff --git a/object-store.h b/object-store.h\nindex 5959d990fc..a3f0d6ac15 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -1,6 +1,8 @@\n #ifndef OBJECT_STORE_H\n #define OBJECT_STORE_H\n \n+#include \"cache.h\"\n+\n struct raw_object_store {\n \t/*\n \t * Path to the repository's object store.\n@@ -8,9 +10,12 @@ struct raw_object_store {\n \t */\n \tchar *objectdir;\n \n+\tstruct alternate_object_database *alt_odb_list;\n+\tstruct alternate_object_database **alt_odb_tail;\n+\n \tunsigned ignore_env : 1;\n };\n-#define RAW_OBJECT_STORE_INIT { NULL, 0 }\n+#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL, 0 }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/packfile.c b/packfile.c\nindex 7dbe8739d1..216ea836ee 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -1,6 +1,7 @@\n #include \"cache.h\"\n #include \"list.h\"\n #include \"pack.h\"\n+#include \"repository.h\"\n #include \"dir.h\"\n #include \"mergesort.h\"\n #include \"packfile.h\"\n@@ -891,7 +892,7 @@ void prepare_packed_git(void)\n \t\treturn;\n \tprepare_packed_git_one(get_object_directory(), 1);\n \tprepare_alt_odb();\n-\tfor (alt = alt_odb_list; alt; alt = alt->next)\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)\n \t\tprepare_packed_git_one(alt->path, 0);\n \trearrange_packed_git();\n \tprepare_packed_git_mru();\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 831d9e7343..1348dce68f 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -22,6 +22,7 @@\n #include \"pack-revindex.h\"\n #include \"sha1-lookup.h\"\n #include \"bulk-checkin.h\"\n+#include \"repository.h\"\n #include \"streaming.h\"\n #include \"dir.h\"\n #include \"list.h\"\n@@ -342,9 +343,6 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,\n \treturn buf->buf;\n }\n \n-struct alternate_object_database *alt_odb_list;\n-static struct alternate_object_database **alt_odb_tail;\n-\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\n@@ -364,7 +362,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)\n \t * Prevent the common mistake of listing the same\n \t * thing twice, or object directory itself.\n \t */\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\tif (!fspathcmp(path->buf, alt->path))\n \t\t\treturn 0;\n \t}\n@@ -424,8 +422,8 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,\n \tent = alloc_alt_odb(pathbuf.buf);\n \n \t/* add the alternate entry */\n-\t*alt_odb_tail = ent;\n-\talt_odb_tail = &(ent->next);\n+\t*the_repository->objects.alt_odb_tail = ent;\n+\tthe_repository->objects.alt_odb_tail = &(ent->next);\n \tent->next = NULL;\n \n \t/* recursively add alternates */\n@@ -559,7 +557,7 @@ void add_to_alternates_file(const char *reference)\n \t\tfprintf_or_die(out, \"%s\\n\", reference);\n \t\tif (commit_lock_file(&lock))\n \t\t\tdie_errno(\"unable to move new alternates file into place\");\n-\t\tif (alt_odb_tail)\n+\t\tif (the_repository->objects.alt_odb_tail)\n \t\t\tlink_alt_odb_entries(reference, '\\n', NULL, 0);\n \t}\n \tfree(alts);\n@@ -657,7 +655,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \tint r = 0;\n \n \tprepare_alt_odb();\n-\tfor (ent = alt_odb_list; ent; ent = ent->next) {\n+\tfor (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {\n \t\tr = fn(ent, cb);\n \t\tif (r)\n \t\t\tbreak;\n@@ -669,12 +667,13 @@ void prepare_alt_odb(void)\n {\n \tconst char *alt;\n \n-\tif (alt_odb_tail)\n+\tif (the_repository->objects.alt_odb_tail)\n \t\treturn;\n \n \talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n \n-\talt_odb_tail = &alt_odb_list;\n+\tthe_repository->objects.alt_odb_tail =\n+\t\t\t&the_repository->objects.alt_odb_list;\n \tlink_alt_odb_entries(alt, PATH_SEP, NULL, 0);\n \n \tread_info_alternates(get_object_directory(), 0);\n@@ -718,7 +717,7 @@ static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)\n {\n \tstruct alternate_object_database *alt;\n \tprepare_alt_odb();\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\tconst char *path = alt_sha1_path(alt, sha1);\n \t\tif (check_and_freshen_file(path, freshen))\n \t\t\treturn 1;\n@@ -878,7 +877,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n \n \tprepare_alt_odb();\n \terrno = ENOENT;\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tif (!lstat(*path, st))\n \t\t\treturn 0;\n@@ -908,7 +907,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \tmost_interesting_errno = errno;\n \n \tprepare_alt_odb();\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tfd = git_open(*path);\n \t\tif (fd >= 0)\ndiff --git a/sha1_name.c b/sha1_name.c\nindex 611c7d24dd..957ce25680 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -10,6 +10,7 @@\n #include \"dir.h\"\n #include \"sha1-array.h\"\n #include \"packfile.h\"\n+#include \"repository.h\"\n \n static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);\n \n@@ -104,7 +105,7 @@ static void find_short_object_filename(struct disambiguate_state *ds)\n \t\t */\n \t\tfakeent = alloc_alt_odb(get_object_directory());\n \t}\n-\tfakeent->next = alt_odb_list;\n+\tfakeent->next = the_repository->objects.alt_odb_list;\n \n \tfor (alt = fakeent; alt && !ds->ambiguous; alt = alt->next) {\n \t\tint pos;\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339547","messageId":"20180216174626.24677-7-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 06/16] pack: move prepare_packed_git_run_once to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:16Z","receivedAt":"2018-02-16T17:46:52Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Each repository's object store can be initialized independently, so\nthey must not share a run_once variable.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 8 +++++++-\n packfile.c | 7 +++----\n 2 files changed, 10 insertions(+), 5 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 024ccc91e9..794cb0af59 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -22,10 +22,16 @@ struct raw_object_store {\n \tstruct alternate_object_database **alt_odb_tail;\n \n \tunsigned ignore_env : 1;\n+\n+\t/*\n+\t * Whether packed_git has already been populated with this repository's\n+\t * packs.\n+\t */\n+\tunsigned packed_git_initialized : 1;\n };\n \n #define MRU_LIST_INIT {NULL, NULL}\n-#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_LIST_INIT, NULL, NULL, 0 }\n+#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_LIST_INIT, NULL, NULL, 0, 0 }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/packfile.c b/packfile.c\nindex 511a2b0cdf..a8a2e7fe43 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -884,12 +884,11 @@ static void prepare_packed_git_mru(void)\n \t\tlist_add_tail(&p->mru, &the_repository->objects.packed_git_mru);\n }\n \n-static int prepare_packed_git_run_once = 0;\n void prepare_packed_git(void)\n {\n \tstruct alternate_object_database *alt;\n \n-\tif (prepare_packed_git_run_once)\n+\tif (the_repository->objects.packed_git_initialized)\n \t\treturn;\n \tprepare_packed_git_one(get_object_directory(), 1);\n \tprepare_alt_odb();\n@@ -897,13 +896,13 @@ void prepare_packed_git(void)\n \t\tprepare_packed_git_one(alt->path, 0);\n \trearrange_packed_git();\n \tprepare_packed_git_mru();\n-\tprepare_packed_git_run_once = 1;\n+\tthe_repository->objects.packed_git_initialized = 1;\n }\n \n void reprepare_packed_git(void)\n {\n \tapproximate_object_count_valid = 0;\n-\tprepare_packed_git_run_once = 0;\n+\tthe_repository->objects.packed_git_initialized = 0;\n \tprepare_packed_git();\n }\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339548","messageId":"20180216174626.24677-11-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 10/16] sha1_file: allow prepare_alt_odb to handle arbitrary object stores","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:20Z","receivedAt":"2018-02-16T17:46:58Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n builtin/fsck.c | 2 +-\n cache.h | 1 -\n object-store.h | 2 ++\n packfile.c | 2 +-\n sha1_file.c | 23 +++++++++++------------\n sha1_name.c | 3 ++-\n 6 files changed, 17 insertions(+), 16 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 2e99e02128..c0d967d6d7 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -717,7 +717,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t} else {\n \t\tfsck_object_dir(get_object_directory());\n \n-\t\tprepare_alt_odb();\n+\t\tprepare_alt_odb(&the_repository->objects);\n \t\tfor (alt = the_repository->objects.alt_odb_list;\n \t\t\t\talt; alt = alt->next)\n \t\t\tfsck_object_dir(alt->path);\ndiff --git a/cache.h b/cache.h\nindex 12bb42f951..d0c1a533c5 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1599,7 +1599,6 @@ struct alternate_object_database {\n \t */\n \tchar path[FLEX_ARRAY];\n };\n-extern void prepare_alt_odb(void);\n extern char *compute_alternate_path(const char *path, struct strbuf *err);\n typedef int alt_odb_fn(struct alternate_object_database *, void *);\n extern int foreach_alt_odb(alt_odb_fn, void*);\ndiff --git a/object-store.h b/object-store.h\nindex 80a2ca6f75..280e8e16ad 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -67,4 +67,6 @@ extern struct packed_git {\n \tchar pack_name[FLEX_ARRAY]; /* more */\n } *packed_git;\n \n+void prepare_alt_odb(struct raw_object_store *o);\n+\n #endif /* OBJECT_STORE_H */\ndiff --git a/packfile.c b/packfile.c\nindex 693bafbc98..83b10e55c0 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -890,7 +890,7 @@ void prepare_packed_git(void)\n \tif (the_repository->objects.packed_git_initialized)\n \t\treturn;\n \tprepare_packed_git_one(get_object_directory(), 1);\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(&the_repository->objects);\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)\n \t\tprepare_packed_git_one(alt->path, 0);\n \trearrange_packed_git();\ndiff --git a/sha1_file.c b/sha1_file.c\nindex c4255a2da6..04a81e29f2 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -23,6 +23,7 @@\n #include \"sha1-lookup.h\"\n #include \"bulk-checkin.h\"\n #include \"repository.h\"\n+#include \"object-store.h\"\n #include \"streaming.h\"\n #include \"dir.h\"\n #include \"list.h\"\n@@ -577,7 +578,7 @@ void add_to_alternates_memory(const char *reference)\n \t * Make sure alternates are initialized, or else our entry may be\n \t * overwritten when they are.\n \t */\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(&the_repository->objects);\n \n \tlink_alt_odb_entries(&the_repository->objects, reference,\n \t\t\t '\\n', NULL, 0);\n@@ -663,7 +664,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \tstruct alternate_object_database *ent;\n \tint r = 0;\n \n-\tprepare_alt_odb();\n+\tprepare_alt_odb(&the_repository->objects);\n \tfor (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {\n \t\tr = fn(ent, cb);\n \t\tif (r)\n@@ -672,21 +673,19 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \treturn r;\n }\n \n-void prepare_alt_odb(void)\n+void prepare_alt_odb(struct raw_object_store *o)\n {\n \tconst char *alt;\n \n-\tif (the_repository->objects.alt_odb_tail)\n+\tif (o->alt_odb_tail)\n \t\treturn;\n \n \talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n \n-\tthe_repository->objects.alt_odb_tail =\n-\t\t\t&the_repository->objects.alt_odb_list;\n-\tlink_alt_odb_entries(&the_repository->objects, alt,\n-\t\t\t PATH_SEP, NULL, 0);\n+\to->alt_odb_tail = &o->alt_odb_list;\n+\tlink_alt_odb_entries(o, alt, PATH_SEP, NULL, 0);\n \n-\tread_info_alternates(&the_repository->objects, get_object_directory(), 0);\n+\tread_info_alternates(o, o->objectdir, 0);\n }\n \n /* Returns 1 if we have successfully freshened the file, 0 otherwise. */\n@@ -726,7 +725,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)\n static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)\n {\n \tstruct alternate_object_database *alt;\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(&the_repository->objects);\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\tconst char *path = alt_sha1_path(alt, sha1);\n \t\tif (check_and_freshen_file(path, freshen))\n@@ -885,7 +884,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n \tif (!lstat(*path, st))\n \t\treturn 0;\n \n-\tprepare_alt_odb();\n+\tprepare_alt_odb(&the_repository->objects);\n \terrno = ENOENT;\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n@@ -916,7 +915,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \t\treturn fd;\n \tmost_interesting_errno = errno;\n \n-\tprepare_alt_odb();\n+\tprepare_alt_odb(&the_repository->objects);\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tfd = git_open(*path);\ndiff --git a/sha1_name.c b/sha1_name.c\nindex 016c883b5c..f1e6c910ba 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -11,6 +11,7 @@\n #include \"sha1-array.h\"\n #include \"packfile.h\"\n #include \"repository.h\"\n+#include \"object-store.h\"\n \n static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);\n \n@@ -353,7 +354,7 @@ static int init_object_disambiguation(const char *name, int len,\n \n \tds->len = len;\n \tds->hex_pfx[len] = '\\0';\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(&the_repository->objects);\n \treturn 0;\n }\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339549","messageId":"20180216174626.24677-13-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 12/16] sha1_file: allow stat_sha1_file to handle arbitrary object stores","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:22Z","receivedAt":"2018-02-16T17:46:59Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 15 ++++++++-------\n 1 file changed, 8 insertions(+), 7 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d64cdbb494..bd8b0331f0 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -871,22 +871,23 @@ int git_open_cloexec(const char *name, int flags)\n * Note that it may point to static storage and is only valid until another\n * call to sha1_file_name(), etc.\n */\n-static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n-\t\t\t const char **path)\n+static int stat_sha1_file(struct raw_object_store *o, const unsigned char *sha1,\n+\t\t\t struct stat *st, const char **path)\n {\n \tstruct alternate_object_database *alt;\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(&the_repository->objects, &buf, sha1);\n+\tsha1_file_name(o, &buf, sha1);\n \t*path = buf.buf;\n \n \tif (!lstat(*path, st))\n \t\treturn 0;\n \n-\tprepare_alt_odb(&the_repository->objects);\n+\tprepare_alt_odb(o);\n+\n \terrno = ENOENT;\n-\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = o->alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tif (!lstat(*path, st))\n \t\t\treturn 0;\n@@ -1178,7 +1179,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,\n \tif (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {\n \t\tconst char *path;\n \t\tstruct stat st;\n-\t\tif (stat_sha1_file(sha1, &st, &path) < 0)\n+\t\tif (stat_sha1_file(&the_repository->objects, sha1, &st, &path) < 0)\n \t\t\treturn -1;\n \t\tif (oi->disk_sizep)\n \t\t\t*oi->disk_sizep = st.st_size;\n@@ -1392,7 +1393,7 @@ void *read_sha1_file_extended(const unsigned char *sha1,\n \t\tdie(\"replacement %s not found for %s\",\n \t\t sha1_to_hex(repl), sha1_to_hex(sha1));\n \n-\tif (!stat_sha1_file(repl, &st, &path))\n+\tif (!stat_sha1_file(&the_repository->objects, repl, &st, &path))\n \t\tdie(\"loose object %s (stored in %s) is corrupt\",\n \t\t sha1_to_hex(repl), path);\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339550","messageId":"20180216174626.24677-15-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 14/16] sha1_file: allow map_sha1_file_1 to handle arbitrary object stores","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:24Z","receivedAt":"2018-02-16T17:47:03Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 +++++------\n 1 file changed, 5 insertions(+), 6 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 3356f70dd2..aeb925a895 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -934,9 +934,8 @@ static int open_sha1_file(struct raw_object_store *o,\n * Map the loose object at \"path\" if it is not NULL, or the path found by\n * searching for a loose object named \"sha1\".\n */\n-static void *map_sha1_file_1(const char *path,\n-\t\t\t const unsigned char *sha1,\n-\t\t\t unsigned long *size)\n+static void *map_sha1_file_1(struct raw_object_store *o, const char *path,\n+\t\t\t const unsigned char *sha1, unsigned long *size)\n {\n \tvoid *map;\n \tint fd;\n@@ -944,7 +943,7 @@ static void *map_sha1_file_1(const char *path,\n \tif (path)\n \t\tfd = git_open(path);\n \telse\n-\t\tfd = open_sha1_file(&the_repository->objects, sha1, &path);\n+\t\tfd = open_sha1_file(o, sha1, &path);\n \tmap = NULL;\n \tif (fd >= 0) {\n \t\tstruct stat st;\n@@ -965,7 +964,7 @@ static void *map_sha1_file_1(const char *path,\n \n void *map_sha1_file(const unsigned char *sha1, unsigned long *size)\n {\n-\treturn map_sha1_file_1(NULL, sha1, size);\n+\treturn map_sha1_file_1(&the_repository->objects, NULL, sha1, size);\n }\n \n static int unpack_sha1_short_header(git_zstream *stream,\n@@ -2195,7 +2194,7 @@ int read_loose_object(const char *path,\n \n \t*contents = NULL;\n \n-\tmap = map_sha1_file_1(path, NULL, &mapsize);\n+\tmap = map_sha1_file_1(&the_repository->objects, path, NULL, &mapsize);\n \tif (!map) {\n \t\terror_errno(\"unable to mmap %s\", path);\n \t\tgoto out;\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339551","messageId":"20180216174626.24677-16-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 15/16] sha1_file: allow map_sha1_file to handle arbitrary object stores","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:25Z","receivedAt":"2018-02-16T17:47:05Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n cache.h | 1 -\n object-store.h | 2 ++\n sha1_file.c | 7 ++++---\n streaming.c | 5 ++++-\n 4 files changed, 10 insertions(+), 5 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 2495c7081d..df29ee72e9 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1242,7 +1242,6 @@ extern int pretend_sha1_file(void *, unsigned long, enum object_type, unsigned c\n extern int force_object_loose(const unsigned char *sha1, time_t mtime);\n extern int git_open_cloexec(const char *name, int flags);\n #define git_open(name) git_open_cloexec(name, O_RDONLY)\n-extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size);\n extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);\n extern int parse_sha1_header(const char *hdr, unsigned long *sizep);\n \ndiff --git a/object-store.h b/object-store.h\nindex 5f6beb9d91..30b4cfdbf5 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -73,6 +73,8 @@ extern struct packed_git {\n */\n void sha1_file_name(struct raw_object_store *o, struct strbuf *buf, const unsigned char *sha1);\n \n+void *map_sha1_file(struct raw_object_store *o, const unsigned char *sha1, unsigned long *size);\n+\n void prepare_alt_odb(struct raw_object_store *o);\n \n #endif /* OBJECT_STORE_H */\ndiff --git a/sha1_file.c b/sha1_file.c\nindex aeb925a895..4cbff471a2 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -962,9 +962,10 @@ static void *map_sha1_file_1(struct raw_object_store *o, const char *path,\n \treturn map;\n }\n \n-void *map_sha1_file(const unsigned char *sha1, unsigned long *size)\n+void *map_sha1_file(struct raw_object_store *o,\n+\t\t const unsigned char *sha1, unsigned long *size)\n {\n-\treturn map_sha1_file_1(&the_repository->objects, NULL, sha1, size);\n+\treturn map_sha1_file_1(o, NULL, sha1, size);\n }\n \n static int unpack_sha1_short_header(git_zstream *stream,\n@@ -1186,7 +1187,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,\n \t\treturn 0;\n \t}\n \n-\tmap = map_sha1_file(sha1, &mapsize);\n+\tmap = map_sha1_file(&the_repository->objects, sha1, &mapsize);\n \tif (!map)\n \t\treturn -1;\n \ndiff --git a/streaming.c b/streaming.c\nindex 5892b50bd8..6888448b12 100644\n--- a/streaming.c\n+++ b/streaming.c\n@@ -3,6 +3,8 @@\n */\n #include \"cache.h\"\n #include \"streaming.h\"\n+#include \"repository.h\"\n+#include \"object-store.h\"\n #include \"packfile.h\"\n \n enum input_source {\n@@ -335,7 +337,8 @@ static struct stream_vtbl loose_vtbl = {\n \n static open_method_decl(loose)\n {\n-\tst->u.loose.mapped = map_sha1_file(sha1, &st->u.loose.mapsize);\n+\tst->u.loose.mapped = map_sha1_file(&the_repository->objects,\n+\t\t\t\t\t sha1, &st->u.loose.mapsize);\n \tif (!st->u.loose.mapped)\n \t\treturn -1;\n \tif ((unpack_sha1_header(&st->z,\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339552","messageId":"20180216174626.24677-14-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 13/16] sha1_file: allow open_sha1_file to handle arbitrary object stores","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:23Z","receivedAt":"2018-02-16T17:47:08Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 ++++++-----\n 1 file changed, 6 insertions(+), 5 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex bd8b0331f0..3356f70dd2 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -900,7 +900,8 @@ static int stat_sha1_file(struct raw_object_store *o, const unsigned char *sha1,\n * Like stat_sha1_file(), but actually open the object and return the\n * descriptor. See the caveats on the \"path\" parameter above.\n */\n-static int open_sha1_file(const unsigned char *sha1, const char **path)\n+static int open_sha1_file(struct raw_object_store *o,\n+\t\t\t const unsigned char *sha1, const char **path)\n {\n \tint fd;\n \tstruct alternate_object_database *alt;\n@@ -908,7 +909,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(&the_repository->objects, &buf, sha1);\n+\tsha1_file_name(o, &buf, sha1);\n \t*path = buf.buf;\n \n \tfd = git_open(*path);\n@@ -916,8 +917,8 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \t\treturn fd;\n \tmost_interesting_errno = errno;\n \n-\tprepare_alt_odb(&the_repository->objects);\n-\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n+\tprepare_alt_odb(o);\n+\tfor (alt = o->alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tfd = git_open(*path);\n \t\tif (fd >= 0)\n@@ -943,7 +944,7 @@ static void *map_sha1_file_1(const char *path,\n \tif (path)\n \t\tfd = git_open(path);\n \telse\n-\t\tfd = open_sha1_file(sha1, &path);\n+\t\tfd = open_sha1_file(&the_repository->objects, sha1, &path);\n \tmap = NULL;\n \tif (fd >= 0) {\n \t\tstruct stat st;\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339553","messageId":"20180216174626.24677-17-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 16/16] sha1_file: allow sha1_loose_object_info to handle arbitrary object stores","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:26Z","receivedAt":"2018-02-16T17:47:11Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n sha1_file.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 4cbff471a2..31be57249f 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -1154,7 +1154,8 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)\n \treturn parse_sha1_header_extended(hdr, &oi, 0);\n }\n \n-static int sha1_loose_object_info(const unsigned char *sha1,\n+static int sha1_loose_object_info(struct raw_object_store *o,\n+\t\t\t\t const unsigned char *sha1,\n \t\t\t\t struct object_info *oi,\n \t\t\t\t int flags)\n {\n@@ -1180,14 +1181,14 @@ static int sha1_loose_object_info(const unsigned char *sha1,\n \tif (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {\n \t\tconst char *path;\n \t\tstruct stat st;\n-\t\tif (stat_sha1_file(&the_repository->objects, sha1, &st, &path) < 0)\n+\t\tif (stat_sha1_file(o, sha1, &st, &path) < 0)\n \t\t\treturn -1;\n \t\tif (oi->disk_sizep)\n \t\t\t*oi->disk_sizep = st.st_size;\n \t\treturn 0;\n \t}\n \n-\tmap = map_sha1_file(&the_repository->objects, sha1, &mapsize);\n+\tmap = map_sha1_file(o, sha1, &mapsize);\n \tif (!map)\n \t\treturn -1;\n \n@@ -1275,7 +1276,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,\n \t\t\tbreak;\n \n \t\t/* Most likely it's a loose object. */\n-\t\tif (!sha1_loose_object_info(real, oi, flags))\n+\t\tif (!sha1_loose_object_info(&the_repository->objects, real, oi, flags))\n \t\t\treturn 0;\n \n \t\t/* Not a loose object; someone else may have just packed it. */\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339554","messageId":"20180216174626.24677-10-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 09/16] sha1_file: allow link_alt_odb_entries to handle arbitrary object stores","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:19Z","receivedAt":"2018-02-16T17:47:13Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Actually this also allows read_info_alternates and link_alt_odb_entry to\nhandle arbitrary object stores, but link_alt_odb_entries is the most\ninteresting function in this set of functions, hence the commit subject.\n\nThese functions span a strongly connected component in the function\ngraph, i.e. the recursive call chain might look like\n\n -> link_alt_odb_entries\n -> link_alt_odb_entry\n -> read_info_alternates\n -> link_alt_odb_entries\n\nThat is why we need to convert all these functions at the same time.\n\nSigned-off-by: Jonathan Nieder \nSigned-off-by: Stefan Beller \n---\n cache.h | 4 ++++\n sha1_file.c | 42 +++++++++++++++++++++++++-----------------\n 2 files changed, 29 insertions(+), 17 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 018a297e10..12bb42f951 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1593,6 +1593,10 @@ struct alternate_object_database {\n \tchar loose_objects_subdir_seen[256];\n \tstruct oid_array loose_objects_cache;\n \n+\t/*\n+\t * Path to the alternative object store. If this is a relative path,\n+\t * it is relative to the current working directory.\n+\t */\n \tchar path[FLEX_ARRAY];\n };\n extern void prepare_alt_odb(void);\ndiff --git a/sha1_file.c b/sha1_file.c\nindex a8e23bd2f8..c4255a2da6 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -389,9 +389,11 @@ static int alt_odb_usable(struct raw_object_store *o,\n * SHA1, an extra slash for the first level indirection, and the\n * terminating NUL.\n */\n-static void read_info_alternates(const char * relative_base, int depth);\n-static int link_alt_odb_entry(const char *entry, const char *relative_base,\n-\tint depth, const char *normalized_objdir)\n+static void read_info_alternates(struct raw_object_store *o,\n+\t\t\t\t const char *relative_base,\n+\t\t\t\t int depth);\n+static int link_alt_odb_entry(struct raw_object_store *o, const char *entry,\n+\tconst char *relative_base, int depth, const char *normalized_objdir)\n {\n \tstruct alternate_object_database *ent;\n \tstruct strbuf pathbuf = STRBUF_INIT;\n@@ -416,7 +418,7 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,\n \twhile (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')\n \t\tstrbuf_setlen(&pathbuf, pathbuf.len - 1);\n \n-\tif (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {\n+\tif (!alt_odb_usable(o, &pathbuf, normalized_objdir)) {\n \t\tstrbuf_release(&pathbuf);\n \t\treturn -1;\n \t}\n@@ -424,12 +426,12 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,\n \tent = alloc_alt_odb(pathbuf.buf);\n \n \t/* add the alternate entry */\n-\t*the_repository->objects.alt_odb_tail = ent;\n-\tthe_repository->objects.alt_odb_tail = &(ent->next);\n+\t*o->alt_odb_tail = ent;\n+\to->alt_odb_tail = &(ent->next);\n \tent->next = NULL;\n \n \t/* recursively add alternates */\n-\tread_info_alternates(pathbuf.buf, depth + 1);\n+\tread_info_alternates(o, pathbuf.buf, depth + 1);\n \n \tstrbuf_release(&pathbuf);\n \treturn 0;\n@@ -464,8 +466,8 @@ static const char *parse_alt_odb_entry(const char *string,\n \treturn end;\n }\n \n-static void link_alt_odb_entries(const char *alt, int sep,\n-\t\t\t\t const char *relative_base, int depth)\n+static void link_alt_odb_entries(struct raw_object_store *o, const char *alt,\n+\t\t\t\t int sep, const char *relative_base, int depth)\n {\n \tstruct strbuf objdirbuf = STRBUF_INIT;\n \tstruct strbuf entry = STRBUF_INIT;\n@@ -479,7 +481,7 @@ static void link_alt_odb_entries(const char *alt, int sep,\n \t\treturn;\n \t}\n \n-\tstrbuf_add_absolute_path(&objdirbuf, get_object_directory());\n+\tstrbuf_add_absolute_path(&objdirbuf, o->objectdir);\n \tif (strbuf_normalize_path(&objdirbuf) < 0)\n \t\tdie(\"unable to normalize object directory: %s\",\n \t\t objdirbuf.buf);\n@@ -488,13 +490,16 @@ static void link_alt_odb_entries(const char *alt, int sep,\n \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n \t\tif (!entry.len)\n \t\t\tcontinue;\n-\t\tlink_alt_odb_entry(entry.buf, relative_base, depth, objdirbuf.buf);\n+\t\tlink_alt_odb_entry(o, entry.buf,\n+\t\t\t\t relative_base, depth, objdirbuf.buf);\n \t}\n \tstrbuf_release(&entry);\n \tstrbuf_release(&objdirbuf);\n }\n \n-static void read_info_alternates(const char * relative_base, int depth)\n+static void read_info_alternates(struct raw_object_store *o,\n+\t\t\t\t const char *relative_base,\n+\t\t\t\t int depth)\n {\n \tchar *path;\n \tstruct strbuf buf = STRBUF_INIT;\n@@ -506,7 +511,7 @@ static void read_info_alternates(const char * relative_base, int depth)\n \t\treturn;\n \t}\n \n-\tlink_alt_odb_entries(buf.buf, '\\n', relative_base, depth);\n+\tlink_alt_odb_entries(o, buf.buf, '\\n', relative_base, depth);\n \tstrbuf_release(&buf);\n \tfree(path);\n }\n@@ -560,7 +565,8 @@ void add_to_alternates_file(const char *reference)\n \t\tif (commit_lock_file(&lock))\n \t\t\tdie_errno(\"unable to move new alternates file into place\");\n \t\tif (the_repository->objects.alt_odb_tail)\n-\t\t\tlink_alt_odb_entries(reference, '\\n', NULL, 0);\n+\t\t\tlink_alt_odb_entries(&the_repository->objects, reference,\n+\t\t\t\t\t '\\n', NULL, 0);\n \t}\n \tfree(alts);\n }\n@@ -573,7 +579,8 @@ void add_to_alternates_memory(const char *reference)\n \t */\n \tprepare_alt_odb();\n \n-\tlink_alt_odb_entries(reference, '\\n', NULL, 0);\n+\tlink_alt_odb_entries(&the_repository->objects, reference,\n+\t\t\t '\\n', NULL, 0);\n }\n \n /*\n@@ -676,9 +683,10 @@ void prepare_alt_odb(void)\n \n \tthe_repository->objects.alt_odb_tail =\n \t\t\t&the_repository->objects.alt_odb_list;\n-\tlink_alt_odb_entries(alt, PATH_SEP, NULL, 0);\n+\tlink_alt_odb_entries(&the_repository->objects, alt,\n+\t\t\t PATH_SEP, NULL, 0);\n \n-\tread_info_alternates(get_object_directory(), 0);\n+\tread_info_alternates(&the_repository->objects, get_object_directory(), 0);\n }\n \n /* Returns 1 if we have successfully freshened the file, 0 otherwise. */\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339555","messageId":"20180216174626.24677-12-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 11/16] sha1_file: allow sha1_file_name to handle arbitrary object stores","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:21Z","receivedAt":"2018-02-16T17:47:16Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"While at it, move the declaration to object-store.h, where it should\nbe easier to find.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n cache.h | 6 ------\n http-walker.c | 3 ++-\n http.c | 5 ++---\n object-store.h | 6 ++++++\n sha1_file.c | 12 ++++++------\n 5 files changed, 16 insertions(+), 16 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex d0c1a533c5..2495c7081d 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -961,12 +961,6 @@ extern void check_repository_format(void);\n #define DATA_CHANGED 0x0020\n #define TYPE_CHANGED 0x0040\n \n-/*\n- * Put in `buf` the name of the file in the local object database that\n- * would be used to store a loose object with the specified sha1.\n- */\n-extern void sha1_file_name(struct strbuf *buf, const unsigned char *sha1);\n-\n /*\n * Return an abbreviated sha1 unique within this repository's object database.\n * The result will be at least `len` characters long, and will be NUL\ndiff --git a/http-walker.c b/http-walker.c\nindex 8bb5d991bb..59bc0417d2 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -1,4 +1,5 @@\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"commit.h\"\n #include \"walker.h\"\n #include \"http.h\"\n@@ -546,7 +547,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1)\n \t\tret = error(\"File %s has bad hash\", hex);\n \t} else if (req->rename < 0) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n-\t\tsha1_file_name(&buf, req->sha1);\n+\t\tsha1_file_name(&the_repository->objects, &buf, req->sha1);\n \t\tret = error(\"unable to write sha1 filename %s\", buf.buf);\n \t\tstrbuf_release(&buf);\n \t}\ndiff --git a/http.c b/http.c\nindex a4a9e583c7..058436d9d3 100644\n--- a/http.c\n+++ b/http.c\n@@ -2247,7 +2247,7 @@ struct http_object_request *new_http_object_request(const char *base_url,\n \thashcpy(freq->sha1, sha1);\n \tfreq->localfile = -1;\n \n-\tsha1_file_name(&filename, sha1);\n+\tsha1_file_name(&the_repository->objects, &filename, sha1);\n \tsnprintf(freq->tmpfile, sizeof(freq->tmpfile),\n \t\t \"%s.temp\", filename.buf);\n \n@@ -2396,8 +2396,7 @@ int finish_http_object_request(struct http_object_request *freq)\n \t\tunlink_or_warn(freq->tmpfile);\n \t\treturn -1;\n \t}\n-\n-\tsha1_file_name(&filename, freq->sha1);\n+\tsha1_file_name(&the_repository->objects, &filename, freq->sha1);\n \tfreq->rename = finalize_object_file(freq->tmpfile, filename.buf);\n \tstrbuf_release(&filename);\n \ndiff --git a/object-store.h b/object-store.h\nindex 280e8e16ad..5f6beb9d91 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -67,6 +67,12 @@ extern struct packed_git {\n \tchar pack_name[FLEX_ARRAY]; /* more */\n } *packed_git;\n \n+/*\n+ * Put in `buf` the name of the file in the local object database that\n+ * would be used to store a loose object with the specified sha1.\n+ */\n+void sha1_file_name(struct raw_object_store *o, struct strbuf *buf, const unsigned char *sha1);\n+\n void prepare_alt_odb(struct raw_object_store *o);\n \n #endif /* OBJECT_STORE_H */\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 04a81e29f2..d64cdbb494 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -323,9 +323,9 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)\n \t}\n }\n \n-void sha1_file_name(struct strbuf *buf, const unsigned char *sha1)\n+void sha1_file_name(struct raw_object_store *o, struct strbuf *buf, const unsigned char *sha1)\n {\n-\tstrbuf_addstr(buf, get_object_directory());\n+\tstrbuf_addstr(buf, o->objectdir);\n \tstrbuf_addch(buf, '/');\n \tfill_sha1_path(buf, sha1);\n }\n@@ -717,7 +717,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(&buf, sha1);\n+\tsha1_file_name(&the_repository->objects, &buf, sha1);\n \n \treturn check_and_freshen_file(buf.buf, freshen);\n }\n@@ -878,7 +878,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(&buf, sha1);\n+\tsha1_file_name(&the_repository->objects, &buf, sha1);\n \t*path = buf.buf;\n \n \tif (!lstat(*path, st))\n@@ -907,7 +907,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(&buf, sha1);\n+\tsha1_file_name(&the_repository->objects, &buf, sha1);\n \t*path = buf.buf;\n \n \tfd = git_open(*path);\n@@ -1592,7 +1592,7 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,\n \tstatic struct strbuf filename = STRBUF_INIT;\n \n \tstrbuf_reset(&filename);\n-\tsha1_file_name(&filename, sha1);\n+\tsha1_file_name(&the_repository->objects, &filename, sha1);\n \n \tfd = create_tmpfile(&tmp_file, filename.buf);\n \tif (fd < 0) {\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339556","messageId":"20180216174626.24677-8-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 07/16] pack: move approximate object count to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:17Z","receivedAt":"2018-02-16T17:47:18Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The approximate_object_count() function maintains a rough count of\nobjects in a repository to estimate how long object name abbreviates\nshould be. Object names are scoped to a repository and the\nappropriate length may differ by repository, so the object count\nshould not be global.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 10 +++++++++-\n packfile.c | 11 +++++------\n 2 files changed, 14 insertions(+), 7 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 794cb0af59..80a2ca6f75 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -23,6 +23,14 @@ struct raw_object_store {\n \n \tunsigned ignore_env : 1;\n \n+\t/*\n+\t * A fast, rough count of the number of objects in the repository.\n+\t * These two fields are not meant for direct access. Use\n+\t * approximate_object_count() instead.\n+\t */\n+\tunsigned long approximate_object_count;\n+\tunsigned approximate_object_count_valid : 1;\n+\n \t/*\n \t * Whether packed_git has already been populated with this repository's\n \t * packs.\n@@ -31,7 +39,7 @@ struct raw_object_store {\n };\n \n #define MRU_LIST_INIT {NULL, NULL}\n-#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_LIST_INIT, NULL, NULL, 0, 0 }\n+#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_LIST_INIT, NULL, NULL, 0, 0, 0, 0 }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/packfile.c b/packfile.c\nindex a8a2e7fe43..693bafbc98 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -803,8 +803,6 @@ static void prepare_packed_git_one(char *objdir, int local)\n \tstrbuf_release(&path);\n }\n \n-static int approximate_object_count_valid;\n-\n /*\n * Give a fast, rough count of the number of objects in the repository. This\n * ignores loose objects completely. If you have a lot of them, then either\n@@ -814,8 +812,8 @@ static int approximate_object_count_valid;\n */\n unsigned long approximate_object_count(void)\n {\n-\tstatic unsigned long count;\n-\tif (!approximate_object_count_valid) {\n+\tif (!the_repository->objects.approximate_object_count_valid) {\n+\t\tunsigned long count;\n \t\tstruct packed_git *p;\n \n \t\tprepare_packed_git();\n@@ -825,8 +823,9 @@ unsigned long approximate_object_count(void)\n \t\t\t\tcontinue;\n \t\t\tcount += p->num_objects;\n \t\t}\n+\t\tthe_repository->objects.approximate_object_count = count;\n \t}\n-\treturn count;\n+\treturn the_repository->objects.approximate_object_count;\n }\n \n static void *get_next_packed_git(const void *p)\n@@ -901,7 +900,7 @@ void prepare_packed_git(void)\n \n void reprepare_packed_git(void)\n {\n-\tapproximate_object_count_valid = 0;\n+\tthe_repository->objects.approximate_object_count_valid = 0;\n \tthe_repository->objects.packed_git_initialized = 0;\n \tprepare_packed_git();\n }\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339557","messageId":"20180216174626.24677-9-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 08/16] sha1_file: add raw_object_store argument to alt_odb_usable","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:18Z","receivedAt":"2018-02-16T17:47:22Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a raw_object_store to alt_odb_usable to be more specific about which\nrepository to act on. The choice of the repository is delegated to its\nonly caller link_alt_odb_entry.\n\nSigned-off-by: Stefan Beller \n---\n sha1_file.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 1348dce68f..a8e23bd2f8 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -346,7 +346,9 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\n-static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)\n+static int alt_odb_usable(struct raw_object_store *o,\n+\t\t\t struct strbuf *path,\n+\t\t\t const char *normalized_objdir)\n {\n \tstruct alternate_object_database *alt;\n \n@@ -362,7 +364,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)\n \t * Prevent the common mistake of listing the same\n \t * thing twice, or object directory itself.\n \t */\n-\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = o->alt_odb_list; alt; alt = alt->next) {\n \t\tif (!fspathcmp(path->buf, alt->path))\n \t\t\treturn 0;\n \t}\n@@ -414,7 +416,7 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,\n \twhile (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')\n \t\tstrbuf_setlen(&pathbuf, pathbuf.len - 1);\n \n-\tif (!alt_odb_usable(&pathbuf, normalized_objdir)) {\n+\tif (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {\n \t\tstrbuf_release(&pathbuf);\n \t\treturn -1;\n \t}\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339558","messageId":"20180216174626.24677-5-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 04/16] object-store: move packed_git and packed_git_mru to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:14Z","receivedAt":"2018-02-16T17:47:27Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a process with multiple repositories open, packfile accessors\nshould be associated to a single repository and not shared globally.\nMove packed_git and packed_git_mru into the_repository and adjust\ncallers to reflect this.\n\nPatch generated by\n\n 1. Moving the struct packed_git declaration to object-store.h\n and packed_git, packed_git_mru globals to struct object_store.\n\n 2. Applying the semantic patch to adjust callers.\n @@ @@\n - packed_git\n + the_repository->objects.packed_git\n\n @@ @@\n - packed_git_mru\n + the_repository->objects.packed_git_mru\n\n 3. Applying line wrapping fixes from \"make style\" to break the\n resulting long lines.\n\n 4. Adding missing #includes of repository.h and object-store.h\n where needed.\n\n 5. As the packfiles are now owned by an objectstore/repository, which\n is ephemeral unlike globals, we introduce memory leaks. So address\n them in raw_object_store_clear().\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/count-objects.c | 6 ++++--\n builtin/fsck.c | 7 +++++--\n builtin/gc.c | 4 +++-\n builtin/index-pack.c | 1 +\n builtin/pack-objects.c | 19 +++++++++++--------\n builtin/pack-redundant.c | 6 ++++--\n builtin/receive-pack.c | 1 +\n cache.h | 29 -----------------------------\n fast-import.c | 6 ++++--\n http-backend.c | 6 ++++--\n http-push.c | 1 +\n http-walker.c | 1 +\n http.c | 1 +\n object-store.h | 36 +++++++++++++++++++++++++++++++++++-\n object.c | 7 +++++++\n pack-bitmap.c | 4 +++-\n pack-check.c | 1 +\n pack-revindex.c | 1 +\n packfile.c | 39 ++++++++++++++++++++-------------------\n reachable.c | 1 +\n repository.c | 3 +++\n server-info.c | 6 ++++--\n sha1_name.c | 5 +++--\n 23 files changed, 118 insertions(+), 73 deletions(-)\n\ndiff --git a/builtin/count-objects.c b/builtin/count-objects.c\nindex 33343818c8..9334648dcc 100644\n--- a/builtin/count-objects.c\n+++ b/builtin/count-objects.c\n@@ -7,6 +7,8 @@\n #include \"cache.h\"\n #include \"config.h\"\n #include \"dir.h\"\n+#include \"repository.h\"\n+#include \"object-store.h\"\n #include \"builtin.h\"\n #include \"parse-options.h\"\n #include \"quote.h\"\n@@ -120,9 +122,9 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)\n \t\tstruct strbuf loose_buf = STRBUF_INIT;\n \t\tstruct strbuf pack_buf = STRBUF_INIT;\n \t\tstruct strbuf garbage_buf = STRBUF_INIT;\n-\t\tif (!packed_git)\n+\t\tif (!the_repository->objects.packed_git)\n \t\t\tprepare_packed_git();\n-\t\tfor (p = packed_git; p; p = p->next) {\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t\tif (!p->pack_local)\n \t\t\t\tcontinue;\n \t\t\tif (open_pack_index(p))\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 908e4f092a..2e99e02128 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -2,6 +2,7 @@\n #include \"cache.h\"\n #include \"repository.h\"\n #include \"config.h\"\n+#include \"object-store.h\"\n #include \"commit.h\"\n #include \"tree.h\"\n #include \"blob.h\"\n@@ -729,7 +730,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t\t\tprepare_packed_git();\n \n \t\t\tif (show_progress) {\n-\t\t\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\t\tfor (p = the_repository->objects.packed_git; p;\n+\t\t\t\t p = p->next) {\n \t\t\t\t\tif (open_pack_index(p))\n \t\t\t\t\t\tcontinue;\n \t\t\t\t\ttotal += p->num_objects;\n@@ -737,7 +739,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \n \t\t\t\tprogress = start_progress(_(\"Checking objects\"), total);\n \t\t\t}\n-\t\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\tfor (p = the_repository->objects.packed_git; p;\n+\t\t\t p = p->next) {\n \t\t\t\t/* verify gives error messages itself */\n \t\t\t\tif (verify_pack(p, fsck_obj_buffer,\n \t\t\t\t\t\tprogress, count))\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 77fa720bd0..96ff790867 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -11,6 +11,7 @@\n */\n \n #include \"builtin.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n #include \"tempfile.h\"\n #include \"lockfile.h\"\n@@ -18,6 +19,7 @@\n #include \"run-command.h\"\n #include \"sigchain.h\"\n #include \"argv-array.h\"\n+#include \"object-store.h\"\n #include \"commit.h\"\n #include \"packfile.h\"\n \n@@ -173,7 +175,7 @@ static int too_many_packs(void)\n \t\treturn 0;\n \n \tprepare_packed_git();\n-\tfor (cnt = 0, p = packed_git; p; p = p->next) {\n+\tfor (cnt = 0, p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (!p->pack_local)\n \t\t\tcontinue;\n \t\tif (p->pack_keep)\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 5ebd370c56..e60cc3ed98 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -12,6 +12,7 @@\n #include \"exec_cmd.h\"\n #include \"streaming.h\"\n #include \"thread-utils.h\"\n+#include \"object-store.h\"\n #include \"packfile.h\"\n \n static const char index_pack_usage[] =\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 83dcbc9773..d73528b583 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1,6 +1,8 @@\n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n+#include \"object-store.h\"\n #include \"attr.h\"\n #include \"object.h\"\n #include \"blob.h\"\n@@ -1026,7 +1028,7 @@ static int want_object_in_pack(const struct object_id *oid,\n \t\t\treturn want;\n \t}\n \n-\tlist_for_each(pos, &packed_git_mru) {\n+\tlist_for_each(pos, &the_repository->objects.packed_git_mru) {\n \t\tstruct packed_git *p = list_entry(pos, struct packed_git, mru);\n \t\toff_t offset;\n \n@@ -1044,7 +1046,7 @@ static int want_object_in_pack(const struct object_id *oid,\n \t\t\t}\n \t\t\twant = want_found_object(exclude, p);\n \t\t\tif (!exclude && want > 0)\n-\t\t\t\tlist_move(&p->mru, &packed_git_mru);\n+\t\t\t\tlist_move(&p->mru, &the_repository->objects.packed_git_mru);\n \t\t\tif (want != -1)\n \t\t\t\treturn want;\n \t\t}\n@@ -2673,7 +2675,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)\n \n \tmemset(&in_pack, 0, sizeof(in_pack));\n \n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tstruct object_id oid;\n \t\tstruct object *o;\n \n@@ -2736,7 +2738,8 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)\n \tstatic struct packed_git *last_found = (void *)1;\n \tstruct packed_git *p;\n \n-\tp = (last_found != (void *)1) ? last_found : packed_git;\n+\tp = (last_found != (void *)1) ? last_found :\n+\t\t\t\t\tthe_repository->objects.packed_git;\n \n \twhile (p) {\n \t\tif ((!p->pack_local || p->pack_keep) &&\n@@ -2745,7 +2748,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)\n \t\t\treturn 1;\n \t\t}\n \t\tif (p == last_found)\n-\t\t\tp = packed_git;\n+\t\t\tp = the_repository->objects.packed_git;\n \t\telse\n \t\t\tp = p->next;\n \t\tif (p == last_found)\n@@ -2781,7 +2784,7 @@ static void loosen_unused_packed_objects(struct rev_info *revs)\n \tuint32_t i;\n \tstruct object_id oid;\n \n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (!p->pack_local || p->pack_keep)\n \t\t\tcontinue;\n \n@@ -3152,7 +3155,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \tprepare_packed_git();\n \tif (ignore_packed_keep) {\n \t\tstruct packed_git *p;\n-\t\tfor (p = packed_git; p; p = p->next)\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\t\tif (p->pack_local && p->pack_keep)\n \t\t\t\tbreak;\n \t\tif (!p) /* no keep-able packs found */\n@@ -3165,7 +3168,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \t\t * also covers non-local objects\n \t\t */\n \t\tstruct packed_git *p;\n-\t\tfor (p = packed_git; p; p = p->next) {\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t\tif (!p->pack_local) {\n \t\t\t\thave_non_local_packs = 1;\n \t\t\t\tbreak;\ndiff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c\nindex aaa8136322..55462bc826 100644\n--- a/builtin/pack-redundant.c\n+++ b/builtin/pack-redundant.c\n@@ -7,6 +7,8 @@\n */\n \n #include \"builtin.h\"\n+#include \"repository.h\"\n+#include \"object-store.h\"\n #include \"packfile.h\"\n \n #define BLKSIZE 512\n@@ -571,7 +573,7 @@ static struct pack_list * add_pack(struct packed_git *p)\n \n static struct pack_list * add_pack_file(const char *filename)\n {\n-\tstruct packed_git *p = packed_git;\n+\tstruct packed_git *p = the_repository->objects.packed_git;\n \n \tif (strlen(filename) < 40)\n \t\tdie(\"Bad pack filename: %s\", filename);\n@@ -586,7 +588,7 @@ static struct pack_list * add_pack_file(const char *filename)\n \n static void load_all(void)\n {\n-\tstruct packed_git *p = packed_git;\n+\tstruct packed_git *p = the_repository->objects.packed_git;\n \n \twhile (p) {\n \t\tadd_pack(p);\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex b7ce7c7f52..b2eac79a6e 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -7,6 +7,7 @@\n #include \"sideband.h\"\n #include \"run-command.h\"\n #include \"exec_cmd.h\"\n+#include \"object-store.h\"\n #include \"commit.h\"\n #include \"object.h\"\n #include \"remote.h\"\ndiff --git a/cache.h b/cache.h\nindex f373c30e25..018a297e10 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1636,35 +1636,6 @@ struct pack_window {\n \tunsigned int inuse_cnt;\n };\n \n-extern struct packed_git {\n-\tstruct packed_git *next;\n-\tstruct list_head mru;\n-\tstruct pack_window *windows;\n-\toff_t pack_size;\n-\tconst void *index_data;\n-\tsize_t index_size;\n-\tuint32_t num_objects;\n-\tuint32_t num_bad_objects;\n-\tunsigned char *bad_object_sha1;\n-\tint index_version;\n-\ttime_t mtime;\n-\tint pack_fd;\n-\tunsigned pack_local:1,\n-\t\t pack_keep:1,\n-\t\t freshened:1,\n-\t\t do_not_close:1,\n-\t\t pack_promisor:1;\n-\tunsigned char sha1[20];\n-\tstruct revindex_entry *revindex;\n-\t/* something like \".git/objects/pack/xxxxx.pack\" */\n-\tchar pack_name[FLEX_ARRAY]; /* more */\n-} *packed_git;\n-\n-/*\n- * A most-recently-used ordered version of the packed_git list.\n- */\n-extern struct list_head packed_git_mru;\n-\n struct pack_entry {\n \toff_t offset;\n \tunsigned char sha1[20];\ndiff --git a/fast-import.c b/fast-import.c\nindex b70ac025e0..1685fe59a2 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -154,8 +154,10 @@ Format of STDIN stream:\n \n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n #include \"lockfile.h\"\n+#include \"object-store.h\"\n #include \"object.h\"\n #include \"blob.h\"\n #include \"tree.h\"\n@@ -1110,7 +1112,7 @@ static int store_object(\n \tif (e->idx.offset) {\n \t\tduplicate_count_by_type[type]++;\n \t\treturn 1;\n-\t} else if (find_sha1_pack(oid.hash, packed_git)) {\n+\t} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {\n \t\te->type = type;\n \t\te->pack_id = MAX_PACK_ID;\n \t\te->idx.offset = 1; /* just not zero! */\n@@ -1305,7 +1307,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)\n \t\tduplicate_count_by_type[OBJ_BLOB]++;\n \t\ttruncate_pack(&checkpoint);\n \n-\t} else if (find_sha1_pack(oid.hash, packed_git)) {\n+\t} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {\n \t\te->type = OBJ_BLOB;\n \t\te->pack_id = MAX_PACK_ID;\n \t\te->idx.offset = 1; /* just not zero! */\ndiff --git a/http-backend.c b/http-backend.c\nindex f3dc218b2a..4d93126c15 100644\n--- a/http-backend.c\n+++ b/http-backend.c\n@@ -1,7 +1,9 @@\n #include \"cache.h\"\n #include \"config.h\"\n+#include \"repository.h\"\n #include \"refs.h\"\n #include \"pkt-line.h\"\n+#include \"object-store.h\"\n #include \"object.h\"\n #include \"tag.h\"\n #include \"exec_cmd.h\"\n@@ -518,13 +520,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg)\n \n \tselect_getanyfile(hdr);\n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (p->pack_local)\n \t\t\tcnt++;\n \t}\n \n \tstrbuf_grow(&buf, cnt * 53 + 2);\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (p->pack_local)\n \t\t\tstrbuf_addf(&buf, \"P %s\\n\", p->pack_name + objdirlen + 6);\n \t}\ndiff --git a/http-push.c b/http-push.c\nindex 0913f8ab86..1ca662390f 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -1,4 +1,5 @@\n #include \"cache.h\"\n+#include \"object-store.h\"\n #include \"commit.h\"\n #include \"tag.h\"\n #include \"blob.h\"\ndiff --git a/http-walker.c b/http-walker.c\nindex 07c2b1af82..8bb5d991bb 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -4,6 +4,7 @@\n #include \"http.h\"\n #include \"list.h\"\n #include \"transport.h\"\n+#include \"object-store.h\"\n #include \"packfile.h\"\n \n struct alt_base {\ndiff --git a/http.c b/http.c\nindex 31755023a4..a4a9e583c7 100644\n--- a/http.c\n+++ b/http.c\n@@ -1,6 +1,7 @@\n #include \"git-compat-util.h\"\n #include \"http.h\"\n #include \"config.h\"\n+#include \"object-store.h\"\n #include \"pack.h\"\n #include \"sideband.h\"\n #include \"run-command.h\"\ndiff --git a/object-store.h b/object-store.h\nindex a3f0d6ac15..024ccc91e9 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -2,6 +2,7 @@\n #define OBJECT_STORE_H\n \n #include \"cache.h\"\n+#include \"list.h\"\n \n struct raw_object_store {\n \t/*\n@@ -10,13 +11,46 @@ struct raw_object_store {\n \t */\n \tchar *objectdir;\n \n+\tstruct packed_git *packed_git;\n+\t/*\n+\t * A most-recently-used ordered version of the packed_git list, which can\n+\t * be iterated instead of packed_git (and marked via mru_mark).\n+\t */\n+\tstruct list_head packed_git_mru;\n+\n \tstruct alternate_object_database *alt_odb_list;\n \tstruct alternate_object_database **alt_odb_tail;\n \n \tunsigned ignore_env : 1;\n };\n-#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL, 0 }\n+\n+#define MRU_LIST_INIT {NULL, NULL}\n+#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_LIST_INIT, NULL, NULL, 0 }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \n+extern struct packed_git {\n+\tstruct packed_git *next;\n+\tstruct list_head mru;\n+\tstruct pack_window *windows;\n+\toff_t pack_size;\n+\tconst void *index_data;\n+\tsize_t index_size;\n+\tuint32_t num_objects;\n+\tuint32_t num_bad_objects;\n+\tunsigned char *bad_object_sha1;\n+\tint index_version;\n+\ttime_t mtime;\n+\tint pack_fd;\n+\tunsigned pack_local:1,\n+\t\t pack_keep:1,\n+\t\t freshened:1,\n+\t\t do_not_close:1,\n+\t\t pack_promisor:1;\n+\tunsigned char sha1[20];\n+\tstruct revindex_entry *revindex;\n+\t/* something like \".git/objects/pack/xxxxx.pack\" */\n+\tchar pack_name[FLEX_ARRAY]; /* more */\n+} *packed_git;\n+\n #endif /* OBJECT_STORE_H */\ndiff --git a/object.c b/object.c\nindex 17b1da6d6b..c76b62572a 100644\n--- a/object.c\n+++ b/object.c\n@@ -466,4 +466,11 @@ void raw_object_store_clear(struct raw_object_store *o)\n \n \tfree_alt_odbs(o);\n \to->alt_odb_tail = NULL;\n+\n+\twhile (!list_empty(&o->packed_git_mru))\n+\t\tlist_del(&o->packed_git_mru);\n+\t/*\n+\t * TODO: call close_all_packs once migrated to\n+\t * take an object store argument\n+\t */\n }\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9270983e5f..d51172b1d5 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -5,11 +5,13 @@\n #include \"revision.h\"\n #include \"progress.h\"\n #include \"list-objects.h\"\n+#include \"object-store.h\"\n #include \"pack.h\"\n #include \"pack-bitmap.h\"\n #include \"pack-revindex.h\"\n #include \"pack-objects.h\"\n #include \"packfile.h\"\n+#include \"repository.h\"\n \n /*\n * An entry on the bitmap index, representing the bitmap for a given\n@@ -335,7 +337,7 @@ static int open_pack_bitmap(void)\n \tassert(!bitmap_git.map && !bitmap_git.loaded);\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (open_pack_bitmap_1(p) == 0)\n \t\t\tret = 0;\n \t}\ndiff --git a/pack-check.c b/pack-check.c\nindex 073c1fbd46..c80c3ed8fd 100644\n--- a/pack-check.c\n+++ b/pack-check.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"pack.h\"\n+#include \"object-store.h\"\n #include \"pack-revindex.h\"\n #include \"progress.h\"\n #include \"packfile.h\"\ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex ff5f62c033..bb521cf7fb 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"pack-revindex.h\"\n+#include \"object-store.h\"\n \n /*\n * Pack index for existing packs give us easy access to the offsets into\ndiff --git a/packfile.c b/packfile.c\nindex 216ea836ee..d41e4c83d0 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -7,6 +7,7 @@\n #include \"packfile.h\"\n #include \"delta.h\"\n #include \"list.h\"\n+#include \"object-store.h\"\n #include \"streaming.h\"\n #include \"sha1-lookup.h\"\n #include \"commit.h\"\n@@ -45,8 +46,6 @@ static unsigned int pack_open_fds;\n static unsigned int pack_max_fds;\n static size_t peak_pack_mapped;\n static size_t pack_mapped;\n-struct packed_git *packed_git;\n-LIST_HEAD(packed_git_mru);\n \n #define SZ_FMT PRIuMAX\n static inline uintmax_t sz_fmt(size_t s) { return s; }\n@@ -246,7 +245,7 @@ static int unuse_one_window(struct packed_git *current)\n \n \tif (current)\n \t\tscan_windows(current, &lru_p, &lru_w, &lru_l);\n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tscan_windows(p, &lru_p, &lru_w, &lru_l);\n \tif (lru_p) {\n \t\tmunmap(lru_w->base, lru_w->len);\n@@ -316,7 +315,7 @@ void close_all_packs(void)\n {\n \tstruct packed_git *p;\n \n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tif (p->do_not_close)\n \t\t\tdie(\"BUG: want to close pack marked 'do-not-close'\");\n \t\telse\n@@ -384,7 +383,7 @@ static int close_one_pack(void)\n \tstruct pack_window *mru_w = NULL;\n \tint accept_windows_inuse = 1;\n \n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (p->pack_fd == -1)\n \t\t\tcontinue;\n \t\tfind_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);\n@@ -686,8 +685,8 @@ void install_packed_git(struct packed_git *pack)\n \tif (pack->pack_fd != -1)\n \t\tpack_open_fds++;\n \n-\tpack->next = packed_git;\n-\tpacked_git = pack;\n+\tpack->next = the_repository->objects.packed_git;\n+\tthe_repository->objects.packed_git = pack;\n }\n \n void (*report_garbage)(unsigned seen_bits, const char *path);\n@@ -769,7 +768,8 @@ static void prepare_packed_git_one(char *objdir, int local)\n \t\tbase_len = path.len;\n \t\tif (strip_suffix_mem(path.buf, &base_len, \".idx\")) {\n \t\t\t/* Don't reopen a pack we already have. */\n-\t\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\tfor (p = the_repository->objects.packed_git; p;\n+\t\t\t p = p->next) {\n \t\t\t\tsize_t len;\n \t\t\t\tif (strip_suffix(p->pack_name, \".pack\", &len) &&\n \t\t\t\t len == base_len &&\n@@ -820,7 +820,7 @@ unsigned long approximate_object_count(void)\n \n \t\tprepare_packed_git();\n \t\tcount = 0;\n-\t\tfor (p = packed_git; p; p = p->next) {\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t\tif (open_pack_index(p))\n \t\t\t\tcontinue;\n \t\t\tcount += p->num_objects;\n@@ -869,18 +869,19 @@ static int sort_pack(const void *a_, const void *b_)\n \n static void rearrange_packed_git(void)\n {\n-\tpacked_git = llist_mergesort(packed_git, get_next_packed_git,\n-\t\t\t\t set_next_packed_git, sort_pack);\n+\tthe_repository->objects.packed_git = llist_mergesort(\n+\t\tthe_repository->objects.packed_git, get_next_packed_git,\n+\t\tset_next_packed_git, sort_pack);\n }\n \n static void prepare_packed_git_mru(void)\n {\n \tstruct packed_git *p;\n \n-\tINIT_LIST_HEAD(&packed_git_mru);\n+\tINIT_LIST_HEAD(&the_repository->objects.packed_git_mru);\n \n-\tfor (p = packed_git; p; p = p->next)\n-\t\tlist_add_tail(&p->mru, &packed_git_mru);\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n+\t\tlist_add_tail(&p->mru, &the_repository->objects.packed_git_mru);\n }\n \n static int prepare_packed_git_run_once = 0;\n@@ -1014,7 +1015,7 @@ const struct packed_git *has_packed_and_bad(const unsigned char *sha1)\n \tstruct packed_git *p;\n \tunsigned i;\n \n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tfor (i = 0; i < p->num_bad_objects; i++)\n \t\t\tif (!hashcmp(sha1, p->bad_object_sha1 + 20 * i))\n \t\t\t\treturn p;\n@@ -1845,13 +1846,13 @@ int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)\n \tstruct list_head *pos;\n \n \tprepare_packed_git();\n-\tif (!packed_git)\n+\tif (!the_repository->objects.packed_git)\n \t\treturn 0;\n \n-\tlist_for_each(pos, &packed_git_mru) {\n+\tlist_for_each(pos, &the_repository->objects.packed_git_mru) {\n \t\tstruct packed_git *p = list_entry(pos, struct packed_git, mru);\n \t\tif (fill_pack_entry(sha1, e, p)) {\n-\t\t\tlist_move(&p->mru, &packed_git_mru);\n+\t\t\tlist_move(&p->mru, &the_repository->objects.packed_git_mru);\n \t\t\treturn 1;\n \t\t}\n \t}\n@@ -1898,7 +1899,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)\n \tint pack_errors = 0;\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)\n \t\t\tcontinue;\n \t\tif ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&\ndiff --git a/reachable.c b/reachable.c\nindex 88d7d679da..59ed16ba76 100644\n--- a/reachable.c\n+++ b/reachable.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"refs.h\"\n+#include \"object-store.h\"\n #include \"tag.h\"\n #include \"commit.h\"\n #include \"blob.h\"\ndiff --git a/repository.c b/repository.c\nindex 3b8f1b91be..61b23558d0 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -144,6 +144,9 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)\n \n \trepo->ignore_env = 1;\n \n+\trepo->objects.ignore_env = 1;\n+\tINIT_LIST_HEAD(&repo->objects.packed_git_mru);\n+\n \tif (repo_init_gitdir(repo, gitdir))\n \t\tgoto error;\n \ndiff --git a/server-info.c b/server-info.c\nindex 26a6c20b7d..96885c45f6 100644\n--- a/server-info.c\n+++ b/server-info.c\n@@ -1,5 +1,7 @@\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"refs.h\"\n+#include \"object-store.h\"\n #include \"object.h\"\n #include \"commit.h\"\n #include \"tag.h\"\n@@ -200,7 +202,7 @@ static void init_pack_info(const char *infofile, int force)\n \tobjdirlen = strlen(objdir);\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t/* we ignore things on alternate path since they are\n \t\t * not available to the pullers in general.\n \t\t */\n@@ -210,7 +212,7 @@ static void init_pack_info(const char *infofile, int force)\n \t}\n \tnum_pack = i;\n \tinfo = xcalloc(num_pack, sizeof(struct pack_info *));\n-\tfor (i = 0, p = packed_git; p; p = p->next) {\n+\tfor (i = 0, p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (!p->pack_local)\n \t\t\tcontinue;\n \t\tinfo[i] = xcalloc(1, sizeof(struct pack_info));\ndiff --git a/sha1_name.c b/sha1_name.c\nindex 957ce25680..016c883b5c 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -196,7 +196,8 @@ static void find_short_packed_object(struct disambiguate_state *ds)\n \tstruct packed_git *p;\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p && !ds->ambiguous; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p && !ds->ambiguous;\n+\t p = p->next)\n \t\tunique_in_pack(p, ds);\n }\n \n@@ -566,7 +567,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)\n \tstruct packed_git *p;\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tfind_abbrev_len_for_pack(p, mad);\n }\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339559","messageId":"20180216174626.24677-6-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCH 05/16] object-store: close all packs upon clearing the object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-16T17:46:15Z","receivedAt":"2018-02-16T17:47:30Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n builtin/am.c | 2 +-\n builtin/clone.c | 2 +-\n builtin/fetch.c | 2 +-\n builtin/merge.c | 2 +-\n builtin/receive-pack.c | 2 +-\n object.c | 6 ++----\n packfile.c | 4 ++--\n packfile.h | 2 +-\n 8 files changed, 10 insertions(+), 12 deletions(-)\n\ndiff --git a/builtin/am.c b/builtin/am.c\nindex 5bdd2d7578..4762a702e3 100644\n--- a/builtin/am.c\n+++ b/builtin/am.c\n@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)\n \t */\n \tif (!state->rebasing) {\n \t\tam_destroy(state);\n-\t\tclose_all_packs();\n+\t\tclose_all_packs(&the_repository->objects);\n \t\trun_command_v_opt(argv_gc_auto, RUN_GIT_CMD);\n \t}\n }\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 101c27a593..13cfaa6488 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -1217,7 +1217,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \ttransport_disconnect(transport);\n \n \tif (option_dissociate) {\n-\t\tclose_all_packs();\n+\t\tclose_all_packs(&the_repository->objects);\n \t\tdissociate_from_references();\n \t}\n \ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 8ee998ea2e..4d72efca78 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -1478,7 +1478,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n \n \tstring_list_clear(&list, 0);\n \n-\tclose_all_packs();\n+\tclose_all_packs(&the_repository->objects);\n \n \targv_array_pushl(&argv_gc_auto, \"gc\", \"--auto\", NULL);\n \tif (verbosity < 0)\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 30264cfd7c..907ae44ab5 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,\n \t\t\t * We ignore errors in 'gc --auto', since the\n \t\t\t * user should see them.\n \t\t\t */\n-\t\t\tclose_all_packs();\n+\t\t\tclose_all_packs(&the_repository->objects);\n \t\t\trun_command_v_opt(argv_gc_auto, RUN_GIT_CMD);\n \t\t}\n \t}\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex b2eac79a6e..954fc72c7c 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -2027,7 +2027,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)\n \t\t\tproc.git_cmd = 1;\n \t\t\tproc.argv = argv_gc_auto;\n \n-\t\t\tclose_all_packs();\n+\t\t\tclose_all_packs(&the_repository->objects);\n \t\t\tif (!start_command(&proc)) {\n \t\t\t\tif (use_sideband)\n \t\t\t\t\tcopy_to_sideband(proc.err, -1, NULL);\ndiff --git a/object.c b/object.c\nindex c76b62572a..34daaf37b3 100644\n--- a/object.c\n+++ b/object.c\n@@ -4,6 +4,7 @@\n #include \"tree.h\"\n #include \"commit.h\"\n #include \"tag.h\"\n+#include \"packfile.h\"\n \n static struct object **obj_hash;\n static int nr_objs, obj_hash_size;\n@@ -469,8 +470,5 @@ void raw_object_store_clear(struct raw_object_store *o)\n \n \twhile (!list_empty(&o->packed_git_mru))\n \t\tlist_del(&o->packed_git_mru);\n-\t/*\n-\t * TODO: call close_all_packs once migrated to\n-\t * take an object store argument\n-\t */\n+\tclose_all_packs(o);\n }\ndiff --git a/packfile.c b/packfile.c\nindex d41e4c83d0..511a2b0cdf 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -311,11 +311,11 @@ static void close_pack(struct packed_git *p)\n \tclose_pack_index(p);\n }\n \n-void close_all_packs(void)\n+void close_all_packs(struct raw_object_store *o)\n {\n \tstruct packed_git *p;\n \n-\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n+\tfor (p = o->packed_git; p; p = p->next)\n \t\tif (p->do_not_close)\n \t\t\tdie(\"BUG: want to close pack marked 'do-not-close'\");\n \t\telse\ndiff --git a/packfile.h b/packfile.h\nindex a7fca598d6..6a2c57045c 100644\n--- a/packfile.h\n+++ b/packfile.h\n@@ -63,7 +63,7 @@ extern void close_pack_index(struct packed_git *);\n \n extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);\n extern void close_pack_windows(struct packed_git *);\n-extern void close_all_packs(void);\n+extern void close_all_packs(struct raw_object_store *o);\n extern void unuse_pack(struct pack_window **);\n extern void clear_delta_base_cache(void);\n extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339576","messageId":"xmqqfu6065ow.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"20180216174626.24677-5-sbeller@google.com","subject":"Re: [PATCH 04/16] object-store: move packed_git and packed_git_mru to object store","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-16T20:27:27Z","receivedAt":"2018-02-16T20:27:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> @@ @@\n> - packed_git_mru\n> + the_repository->objects.packed_git_mru\n\nRegarding this...\n\n> diff --git a/object-store.h b/object-store.h\n> index a3f0d6ac15..024ccc91e9 100644\n> --- a/object-store.h\n> +++ b/object-store.h\n> @@ -2,6 +2,7 @@\n> #define OBJECT_STORE_H\n> \n> #include \"cache.h\"\n> +#include \"list.h\"\n> \n> struct raw_object_store {\n> ... \n> +\tstruct packed_git *packed_git;\n> +\t/*\n> +\t * A most-recently-used ordered version of the packed_git list, which can\n> +\t * be iterated instead of packed_git (and marked via mru_mark).\n> +\t */\n> +\tstruct list_head packed_git_mru;\n> +\n> \tstruct alternate_object_database *alt_odb_list;\n> \tstruct alternate_object_database **alt_odb_tail;\n> \n> \tunsigned ignore_env : 1;\n> };\n> -#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL, 0 }\n> +\n> +#define MRU_LIST_INIT {NULL, NULL}\n> +#define RAW_OBJECT_STORE_INIT { NULL, NULL, MRU_LIST_INIT, NULL, NULL, 0 }\n> ...\n> diff --git a/packfile.c b/packfile.c\n> index 216ea836ee..d41e4c83d0 100644\n> --- a/packfile.c\n> +++ b/packfile.c\n> @@ -7,6 +7,7 @@\n> -...\n> -LIST_HEAD(packed_git_mru);\n\nGiven that the definition of LIST_HEAD() is\n\n /* Define a variable with the head and tail of the list. */\n #define LIST_HEAD(name) \\\n struct list_head name = { &(name), &(name) }\n\ndoesn't the updated definition of RAW_OBJECT_STORE_INIT look fishy?\n\n"},{"id":"339584","messageId":"20180216223404.GD216564@aiede.svl.corp.google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2018-02-16T22:34:04Z","receivedAt":"2018-02-16T22:34:13Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nStefan Beller wrote:\n\n> v2:\n> * duplicated the 'ignore_env' bit into the object store as well\n> * the #define trick no longer works as we do not have a \"the_objectstore\" global,\n> which means there is just one patch per function that is converted.\n> As this follows the same structure of the previous series, I am still confident\n> there is no hidden dependencies to globals outside the object store in these\n> converted functions.\n> * rebased on top of current master, resolving the merge conflicts.\n> I think I used the list.h APIs right, but please double check.\n\nFor what it's worth, I think I prefer v1. I put some comments on why\non patch 0 of v1 and would be interested in your thoughts on them\n(e.g. as a reply to that). I also think that even if we want to\nswitch to a style that passes around object_store separately from\nrepository, it is easier to do the migration in two steps: first get\nrid of hidden dependencies on the_repository, then do the (simpler)\nautomatic migration from\n\n f(the_repository)\n\nto\n\n f(the_repository->object_store)\n\n*afterwards*.\n\nThoughts?\n\nThanks,\nJonathan\n"},{"id":"339722","messageId":"CAGZ79kafB8JZ0hz4ZbjSpkA2j2AOWjFjoxcQ3e9KcovLp1tzxw@mail.gmail.com","threadId":"47823","inReplyTo":"20180216223404.GD216564@aiede.svl.corp.google.com","subject":"Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-20T18:55:35Z","receivedAt":"2018-02-20T18:55:41Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Feb 16, 2018 at 2:34 PM, Jonathan Nieder wrote:\n> Hi,\n>\n> Stefan Beller wrote:\n>\n>> v2:\n>> * duplicated the 'ignore_env' bit into the object store as well\n>> * the #define trick no longer works as we do not have a \"the_objectstore\" global,\n>> which means there is just one patch per function that is converted.\n>> As this follows the same structure of the previous series, I am still confident\n>> there is no hidden dependencies to globals outside the object store in these\n>> converted functions.\n>> * rebased on top of current master, resolving the merge conflicts.\n>> I think I used the list.h APIs right, but please double check.\n>\n> For what it's worth, I think I prefer v1. I put some comments on why\n> on patch 0 of v1 and would be interested in your thoughts on them\n> (e.g. as a reply to that). I also think that even if we want to\n> switch to a style that passes around object_store separately from\n> repository, it is easier to do the migration in two steps: first get\n> rid of hidden dependencies on the_repository, then do the (simpler)\n> automatic migration from\n>\n> f(the_repository)\n>\n> to\n>\n> f(the_repository->object_store)\n>\n> *afterwards*.\n>\n> Thoughts?\n\nI would prefer to not spend more time on these conversions.\nIf Duy and you would come to a conclusion to either pick this\nor the previous version I would be happy.\n\nI do not see the benefit in splitting up the series even further and\ndo this multistep f(repo) -> f(object store), as the cost in potential\nmerge conflicts is too high. Note that brian just sent another object\nid conversion series, also touching sha1_file.c, which I am sure will\nproduce merge conflicts for Junio.\n\nFor the next part 2 and onwards I'd be happy to take either this\nstrategy or Duys strategy as requested.\n\nStefan\n"},{"id":"339723","messageId":"20180220190009.GA201604@google.com","threadId":"47823","inReplyTo":"CAGZ79kafB8JZ0hz4ZbjSpkA2j2AOWjFjoxcQ3e9KcovLp1tzxw@mail.gmail.com","subject":"Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-20T19:00:09Z","receivedAt":"2018-02-20T19:00:20Z","isPatch":false,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/20, Stefan Beller wrote:\n> On Fri, Feb 16, 2018 at 2:34 PM, Jonathan Nieder wrote:\n> > Hi,\n> >\n> > Stefan Beller wrote:\n> >\n> >> v2:\n> >> * duplicated the 'ignore_env' bit into the object store as well\n> >> * the #define trick no longer works as we do not have a \"the_objectstore\" global,\n> >> which means there is just one patch per function that is converted.\n> >> As this follows the same structure of the previous series, I am still confident\n> >> there is no hidden dependencies to globals outside the object store in these\n> >> converted functions.\n> >> * rebased on top of current master, resolving the merge conflicts.\n> >> I think I used the list.h APIs right, but please double check.\n> >\n> > For what it's worth, I think I prefer v1. I put some comments on why\n> > on patch 0 of v1 and would be interested in your thoughts on them\n> > (e.g. as a reply to that). I also think that even if we want to\n> > switch to a style that passes around object_store separately from\n> > repository, it is easier to do the migration in two steps: first get\n> > rid of hidden dependencies on the_repository, then do the (simpler)\n> > automatic migration from\n> >\n> > f(the_repository)\n> >\n> > to\n> >\n> > f(the_repository->object_store)\n> >\n> > *afterwards*.\n> >\n> > Thoughts?\n> \n> I would prefer to not spend more time on these conversions.\n> If Duy and you would come to a conclusion to either pick this\n> or the previous version I would be happy.\n> \n> I do not see the benefit in splitting up the series even further and\n> do this multistep f(repo) -> f(object store), as the cost in potential\n> merge conflicts is too high. Note that brian just sent another object\n> id conversion series, also touching sha1_file.c, which I am sure will\n> produce merge conflicts for Junio.\n> \n> For the next part 2 and onwards I'd be happy to take either this\n> strategy or Duys strategy as requested.\n\nI think Jonathan is trying to point out that converting to f(repo) maybe\neasier than converting to f(repo->object_store) upfront which would make\nit easier to write the patches (which most of them are already written)\nand to review because you can use the #define trick to make some sort of\nguarantees.\n\nAfter we have successfully completed the migration to f(repo), then we\ncan revisit the subsystems which want to have a clearer abstraction\nlayer and make the jump to f(repo->object_store).\n\n-- \nBrandon Williams\n"},{"id":"339724","messageId":"xmqq371v4h6u.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"20180216223404.GD216564@aiede.svl.corp.google.com","subject":"Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-20T19:03:21Z","receivedAt":"2018-02-20T19:03:28Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder writes:\n\n> For what it's worth, I think I prefer v1. I put some comments on why\n> on patch 0 of v1 and would be interested in your thoughts on them\n> (e.g. as a reply to that). I also think that even if we want to\n> switch to a style that passes around object_store separately from\n> repository, it is easier to do the migration in two steps: first get\n> rid of hidden dependencies on the_repository, then do the (simpler)\n> automatic migration from\n>\n> f(the_repository)\n>\n> to\n>\n> f(the_repository->object_store)\n>\n> *afterwards*.\n>\n> Thoughts?\n\nAre we envisioning the future in which one repository has more than\none object-store (I am counting an object store and its alternates\nthat are pointed by its $GIT_OBJECT_DIRECTORY/info/alternates as a\nsingle logical \"object store\")? If not, doing f(the_repository)\nmigration, stopping there without f(the_repository->object_store)\nmay perfectly be adequate, I would think.\n\n\n"},{"id":"339725","messageId":"CAGZ79kZaDieXu+O6zA1W1pciMqa5_enjNXr9MOuzn=3oDSoU-w@mail.gmail.com","threadId":"47823","inReplyTo":"20180220190009.GA201604@google.com","subject":"Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-20T19:03:48Z","receivedAt":"2018-02-20T19:03:53Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Feb 20, 2018 at 11:00 AM, Brandon Williams wrote:\n> On 02/20, Stefan Beller wrote:\n>> On Fri, Feb 16, 2018 at 2:34 PM, Jonathan Nieder wrote:\n>> > Hi,\n>> >\n>> > Stefan Beller wrote:\n>> >\n>> >> v2:\n>> >> * duplicated the 'ignore_env' bit into the object store as well\n>> >> * the #define trick no longer works as we do not have a \"the_objectstore\" global,\n>> >> which means there is just one patch per function that is converted.\n>> >> As this follows the same structure of the previous series, I am still confident\n>> >> there is no hidden dependencies to globals outside the object store in these\n>> >> converted functions.\n>> >> * rebased on top of current master, resolving the merge conflicts.\n>> >> I think I used the list.h APIs right, but please double check.\n>> >\n>> > For what it's worth, I think I prefer v1. I put some comments on why\n>> > on patch 0 of v1 and would be interested in your thoughts on them\n>> > (e.g. as a reply to that). I also think that even if we want to\n>> > switch to a style that passes around object_store separately from\n>> > repository, it is easier to do the migration in two steps: first get\n>> > rid of hidden dependencies on the_repository, then do the (simpler)\n>> > automatic migration from\n>> >\n>> > f(the_repository)\n>> >\n>> > to\n>> >\n>> > f(the_repository->object_store)\n>> >\n>> > *afterwards*.\n>> >\n>> > Thoughts?\n>>\n>> I would prefer to not spend more time on these conversions.\n>> If Duy and you would come to a conclusion to either pick this\n>> or the previous version I would be happy.\n>>\n>> I do not see the benefit in splitting up the series even further and\n>> do this multistep f(repo) -> f(object store), as the cost in potential\n>> merge conflicts is too high. Note that brian just sent another object\n>> id conversion series, also touching sha1_file.c, which I am sure will\n>> produce merge conflicts for Junio.\n>>\n>> For the next part 2 and onwards I'd be happy to take either this\n>> strategy or Duys strategy as requested.\n>\n> I think Jonathan is trying to point out that converting to f(repo) maybe\n> easier than converting to f(repo->object_store) upfront\n\nI agree.\n\n> which would make\n> it easier to write the patches (which most of them are already written)\n\ntrue, but for this series we also have the conversion to f(object_store)\nwritten already.\n\n> and to review because you can use the #define trick to make some sort of\n> guarantees.\n\nThat is true, so it would be a tradeoff between reviewers and maintainers time?\n\n> After we have successfully completed the migration to f(repo), then we\n> can revisit the subsystems which want to have a clearer abstraction\n> layer and make the jump to f(repo->object_store).\n\nI would think we can take this series as-is and then move on with making\nf(repo) abstractions, eventually moving to f(specialized-subsystem) as those\npatches are not written yet (neither direct conversions, nor the repo trick;\nthe patches I already have need adaption which takes enough time on its own.)\n\n\n>\n> --\n> Brandon Williams\n"},{"id":"339727","messageId":"CAGZ79kaUYYqtaJ-S6WaQ9NmXDnf6_wY_fRCSxLR8=Do1qSLMug@mail.gmail.com","threadId":"47823","inReplyTo":"xmqq371v4h6u.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-20T19:06:25Z","receivedAt":"2018-02-20T19:06:31Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Feb 20, 2018 at 11:03 AM, Junio C Hamano wrote:\n> Jonathan Nieder writes:\n>\n>> For what it's worth, I think I prefer v1. I put some comments on why\n>> on patch 0 of v1 and would be interested in your thoughts on them\n>> (e.g. as a reply to that). I also think that even if we want to\n>> switch to a style that passes around object_store separately from\n>> repository, it is easier to do the migration in two steps: first get\n>> rid of hidden dependencies on the_repository, then do the (simpler)\n>> automatic migration from\n>>\n>> f(the_repository)\n>>\n>> to\n>>\n>> f(the_repository->object_store)\n>>\n>> *afterwards*.\n>>\n>> Thoughts?\n>\n> Are we envisioning the future in which one repository has more than\n> one object-store (I am counting an object store and its alternates\n> that are pointed by its $GIT_OBJECT_DIRECTORY/info/alternates as a\n> single logical \"object store\")? If not, doing f(the_repository)\n> migration, stopping there without f(the_repository->object_store)\n> may perfectly be adequate, I would think.\n>\n\nI do not envision that, maybe Christian Couder does?\nThe partial clone world would be happy with just one object store\nper repo, I would think.\n\nThe step to take an object store would just add expressiveness\nto the code, which may help in understanding what part of the code is\nrelated to what other part of the code, so it may be a readability gain\non its own?\n"},{"id":"339734","messageId":"xmqqk1v7307r.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"CAGZ79kaUYYqtaJ-S6WaQ9NmXDnf6_wY_fRCSxLR8=Do1qSLMug@mail.gmail.com","subject":"Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-20T19:55:20Z","receivedAt":"2018-02-20T19:55:28Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> The step to take an object store would just add expressiveness\n> to the code, which may help in understanding what part of the code is\n> related to what other part of the code, so it may be a readability gain\n> on its own?\n\nIt certainly would allow you to have a standalone object store that\nis not associated with *any* repository, but if we are not using\nsuch a layout, I doubt it would be a readability gain to add excess\nand unexercised expressiveness to the code.\n\n"},{"id":"339736","messageId":"CAGZ79kbyB49J8eT0oQoA0UCzRnn=gBdiSZy3mB03gJaihVGTvQ@mail.gmail.com","threadId":"47823","inReplyTo":"xmqqk1v7307r.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCHv2 00/16] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-20T20:16:33Z","receivedAt":"2018-02-20T20:16:40Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Feb 20, 2018 at 11:55 AM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> The step to take an object store would just add expressiveness\n>> to the code, which may help in understanding what part of the code is\n>> related to what other part of the code, so it may be a readability gain\n>> on its own?\n>\n> It certainly would allow you to have a standalone object store that\n> is not associated with *any* repository, but if we are not using\n> such a layout, I doubt it would be a readability gain to add excess\n> and unexercised expressiveness to the code.\n\nSo you favor v1?\nDuy seems to be ok with v1 too if there is consensus that it is best\n(or rather \"if it makes Stefan's life hell, it's not worth doing.\")[1].\n\nI will try to resend that v1 shortly[2], as the only actual concern about the\ncode was where one struct was defined[3]. All other discussion was\nmeta-level, which direction to go.\n\nThanks for the clarification!\nStefan\n\n[1] https://public-inbox.org/git/CACsJy8CPKESE8atc_eWdNVknQYp9T6ebwKwCdzLHyaFKH2BnZA@mail.gmail.com/\n[2] https://public-inbox.org/git/20180213012241.187007-1-sbeller@google.com/\n[3] https://public-inbox.org/git/20180213185120.GA108526@google.com/\n"},{"id":"339765","messageId":"20180221015430.96054-1-sbeller@google.com","threadId":"47823","inReplyTo":"20180216174626.24677-1-sbeller@google.com","subject":"[PATCHv3 00/27] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:03Z","receivedAt":"2018-02-21T01:54:46Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"v3:\n* reverted back to use the repository for most of the functions\n (including unduplicating 'ignore_env')\n* rebased on master again (I lost that state when doing v2, as\n I did both rebase as well as conversion to object store in one go)\n* one additional patch, that moves the alternates related things to\n object-store.h, such that inclusion of cache.h (in object-store.h)\n is not required any more.\n\nv2:\n* duplicated the 'ignore_env' bit into the object store as well\n* the #define trick no longer works as we do not have a \"the_objectstore\" global,\n which means there is just one patch per function that is converted.\n As this follows the same structure of the previous series, I am still confident\n there is no hidden dependencies to globals outside the object store in these\n converted functions.\n* rebased on top of current master, resolving the merge conflicts.\n I think I used the list.h APIs right, but please double check.\n\nThanks,\nStefan\n\nv1:\nThis is a real take on the first part of the recent RFC[1].\n\nJonathan Tan suggested[2] that \"sha1_loose_object_info to handle arbitrary repositories\"\nmight be a good breaking point for a first part at that RFC at patch 38.\nThis series is smaller and contains only 26 patches as the patches in the big\nRFC were slightly out of order.\n\nI developed this series partly by writing patches, but mostly by cherrypicking\nfrom that RFC on top of current master. I noticed no external conflicts apart\nfrom one addition to the repositories _INIT macro, which was easy to resolve.\n\nComments in the early range of that RFC were on 003 where Junio pointed out\nthat the coccinelle patch ought to be not in contrib/coccinelle, so I put it\nin a sub directory there, as 'make coccicheck' doesn't traverse subdirs.\n\nbrian had a questoin on patch 25 in the RFC, but that seemed to resolve itself\nwithout any suggestion to include into this series[3].\n\nDuy suggested that we shall not use the repository blindly, but should carefully\nexamine whether to pass on an object store or the refstore or such[4], which\nI agree with if it makes sense. This series unfortunately has an issue with that\nas I would not want to pass down the `ignore_env` flag separately from the object\nstore, so I made all functions that only take the object store to have the raw\nobject store as the first parameter, and others using the full repository.\n\nEric Sunshine brought up memory leaks with the RFC, and I would think to\nhave plugged all holes.\n\n[1] https://public-inbox.org/git/20180205235508.216277-1-sbeller@google.com/\n[2] https://public-inbox.org/git/20180207143300.ce1c39ca07f6a0d64fe0e7ca@google.com/\n[3] https://public-inbox.org/git/20180206011940.GD7904@genre.crustytoothpaste.net/\n[4] https://public-inbox.org/git/CACsJy8CGgekpX4cZkyyTSPrj87uQVKZSOL7fyT__P2dh_1LmVQ@mail.gmail.com/\n\nThanks,\nStefan\n\nJonathan Nieder (2):\n sha1_file: allow map_sha1_file_1 to handle arbitrary repositories\n sha1_file: allow sha1_loose_object_info to handle arbitrary\n repositories\n\nStefan Beller (25):\n repository: introduce raw object store field\n object-store: migrate alternates struct and functions from cache.h\n object-store: move alt_odb_list and alt_odb_tail to object store\n object-store: free alt_odb_list\n object-store: move packed_git and packed_git_mru to object store\n object-store: close all packs upon clearing the object store\n pack: move prepare_packed_git_run_once to object store\n pack: move approximate object count to object store\n sha1_file: add raw_object_store argument to alt_odb_usable\n sha1_file: add repository argument to link_alt_odb_entry\n sha1_file: add repository argument to read_info_alternates\n sha1_file: add repository argument to link_alt_odb_entries\n sha1_file: add repository argument to prepare_alt_odb\n sha1_file: allow link_alt_odb_entries to handle arbitrary repositories\n sha1_file: allow prepare_alt_odb to handle arbitrary repositories\n sha1_file: add repository argument to sha1_file_name\n sha1_file: add repository argument to stat_sha1_file\n sha1_file: add repository argument to open_sha1_file\n sha1_file: add repository argument to map_sha1_file_1\n sha1_file: add repository argument to map_sha1_file\n sha1_file: add repository argument to sha1_loose_object_info\n sha1_file: allow sha1_file_name to handle arbitrary repositories\n sha1_file: allow stat_sha1_file to handle arbitrary repositories\n sha1_file: allow open_sha1_file to handle arbitrary repositories\n sha1_file: allow map_sha1_file to handle arbitrary repositories\n\n builtin/am.c | 2 +-\n builtin/clone.c | 2 +-\n builtin/count-objects.c | 6 +-\n builtin/fetch.c | 2 +-\n builtin/fsck.c | 13 ++--\n builtin/gc.c | 4 +-\n builtin/grep.c | 2 +-\n builtin/index-pack.c | 1 +\n builtin/merge.c | 2 +-\n builtin/pack-objects.c | 20 +++---\n builtin/pack-redundant.c | 6 +-\n builtin/receive-pack.c | 3 +-\n cache.h | 87 -------------------------\n environment.c | 5 +-\n fast-import.c | 6 +-\n http-backend.c | 6 +-\n http-push.c | 1 +\n http-walker.c | 4 +-\n http.c | 6 +-\n object-store.h | 134 +++++++++++++++++++++++++++++++++++++++\n object.c | 27 ++++++++\n pack-bitmap.c | 4 +-\n pack-check.c | 1 +\n pack-revindex.c | 1 +\n packfile.c | 64 +++++++++----------\n packfile.h | 2 +-\n path.c | 2 +-\n reachable.c | 1 +\n repository.c | 19 ++++--\n repository.h | 7 +-\n server-info.c | 6 +-\n sha1_file.c | 134 +++++++++++++++++++++------------------\n sha1_name.c | 11 ++--\n streaming.c | 5 +-\n 34 files changed, 365 insertions(+), 231 deletions(-)\n create mode 100644 object-store.h\n\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339766","messageId":"20180221015430.96054-2-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 01/27] repository: introduce raw object store field","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:04Z","receivedAt":"2018-02-21T01:54:48Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The raw object store field will contain any objects needed for\naccess to objects in a given repository.\n\nThis patch introduces the raw object store and populates it with the\n`objectdir`, which used to be part of the repository struct.\n\nAs the struct gains members, we'll also populate the function to clear\nthe memory for these members.\n\nIn a later we'll introduce a struct object_parser, that will complement\nthe object parsing in a repository struct: The raw object parser is the\nlayer that will provide access to raw object content, while the higher\nlevel object parser code will parse raw objects and keeps track of\nparenthood and other object relationships using 'struct object'.\nFor now only add the lower level to the repository struct.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/grep.c | 2 +-\n environment.c | 5 +++--\n object-store.h | 15 +++++++++++++++\n object.c | 5 +++++\n path.c | 2 +-\n repository.c | 17 +++++++++++++----\n repository.h | 7 ++++---\n 7 files changed, 42 insertions(+), 11 deletions(-)\n create mode 100644 object-store.h\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 3ca4ac80d8..0f0c195705 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -432,7 +432,7 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject,\n \t * object.\n \t */\n \tgrep_read_lock();\n-\tadd_to_alternates_memory(submodule.objectdir);\n+\tadd_to_alternates_memory(submodule.objects.objectdir);\n \tgrep_read_unlock();\n \n \tif (oid) {\ndiff --git a/environment.c b/environment.c\nindex de8431e01e..ec10b062e6 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -13,6 +13,7 @@\n #include \"refs.h\"\n #include \"fmt-merge-msg.h\"\n #include \"commit.h\"\n+#include \"object-store.h\"\n \n int trust_executable_bit = 1;\n int trust_ctime = 1;\n@@ -244,9 +245,9 @@ const char *get_git_work_tree(void)\n \n char *get_object_directory(void)\n {\n-\tif (!the_repository->objectdir)\n+\tif (!the_repository->objects.objectdir)\n \t\tBUG(\"git environment hasn't been setup\");\n-\treturn the_repository->objectdir;\n+\treturn the_repository->objects.objectdir;\n }\n \n int odb_mkstemp(struct strbuf *template, const char *pattern)\ndiff --git a/object-store.h b/object-store.h\nnew file mode 100644\nindex 0000000000..cf35760ceb\n--- /dev/null\n+++ b/object-store.h\n@@ -0,0 +1,15 @@\n+#ifndef OBJECT_STORE_H\n+#define OBJECT_STORE_H\n+\n+struct raw_object_store {\n+\t/*\n+\t * Path to the repository's object store.\n+\t * Cannot be NULL after initialization.\n+\t */\n+\tchar *objectdir;\n+};\n+#define RAW_OBJECT_STORE_INIT { NULL }\n+\n+void raw_object_store_clear(struct raw_object_store *o);\n+\n+#endif /* OBJECT_STORE_H */\ndiff --git a/object.c b/object.c\nindex 9e6f9ff20b..11d904c033 100644\n--- a/object.c\n+++ b/object.c\n@@ -445,3 +445,8 @@ void clear_commit_marks_all(unsigned int flags)\n \t\t\tobj->flags &= ~flags;\n \t}\n }\n+\n+void raw_object_store_clear(struct raw_object_store *o)\n+{\n+\tfree(o->objectdir);\n+}\ndiff --git a/path.c b/path.c\nindex da8b655730..81a42d9115 100644\n--- a/path.c\n+++ b/path.c\n@@ -382,7 +382,7 @@ static void adjust_git_path(const struct repository *repo,\n \t\tstrbuf_splice(buf, 0, buf->len,\n \t\t\t repo->index_file, strlen(repo->index_file));\n \telse if (dir_prefix(base, \"objects\"))\n-\t\treplace_dir(buf, git_dir_len + 7, repo->objectdir);\n+\t\treplace_dir(buf, git_dir_len + 7, repo->objects.objectdir);\n \telse if (git_hooks_path && dir_prefix(base, \"hooks\"))\n \t\treplace_dir(buf, git_dir_len + 5, git_hooks_path);\n \telse if (repo->different_commondir)\ndiff --git a/repository.c b/repository.c\nindex 4ffbe9bc94..2255ff657e 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -1,11 +1,18 @@\n #include \"cache.h\"\n #include \"repository.h\"\n+#include \"object-store.h\"\n #include \"config.h\"\n #include \"submodule-config.h\"\n \n /* The main repository */\n static struct repository the_repo = {\n-\tNULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0\n+\tNULL, NULL,\n+\tRAW_OBJECT_STORE_INIT,\n+\tNULL, NULL, NULL,\n+\tNULL, NULL, NULL,\n+\t&the_index,\n+\t&hash_algos[GIT_HASH_SHA1],\n+\t0, 0\n };\n struct repository *the_repository = &the_repo;\n \n@@ -42,8 +49,8 @@ static void repo_setup_env(struct repository *repo)\n \t\t\t\t\t\t !repo->ignore_env);\n \tfree(repo->commondir);\n \trepo->commondir = strbuf_detach(&sb, NULL);\n-\tfree(repo->objectdir);\n-\trepo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,\n+\tfree(repo->objects.objectdir);\n+\trepo->objects.objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,\n \t\t\t\t\t \"objects\", !repo->ignore_env);\n \tfree(repo->graft_file);\n \trepo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,\n@@ -209,12 +216,14 @@ void repo_clear(struct repository *repo)\n {\n \tFREE_AND_NULL(repo->gitdir);\n \tFREE_AND_NULL(repo->commondir);\n-\tFREE_AND_NULL(repo->objectdir);\n \tFREE_AND_NULL(repo->graft_file);\n \tFREE_AND_NULL(repo->index_file);\n \tFREE_AND_NULL(repo->worktree);\n \tFREE_AND_NULL(repo->submodule_prefix);\n \n+\traw_object_store_clear(&repo->objects);\n+\tmemset(&repo->objects, 0, sizeof(repo->objects));\n+\n \tif (repo->config) {\n \t\tgit_configset_clear(repo->config);\n \t\tFREE_AND_NULL(repo->config);\ndiff --git a/repository.h b/repository.h\nindex 0329e40c7f..1f8bc7a7cf 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -1,6 +1,8 @@\n #ifndef REPOSITORY_H\n #define REPOSITORY_H\n \n+#include \"object-store.h\"\n+\n struct config_set;\n struct index_state;\n struct submodule_cache;\n@@ -21,10 +23,9 @@ struct repository {\n \tchar *commondir;\n \n \t/*\n-\t * Path to the repository's object store.\n-\t * Cannot be NULL after initialization.\n+\t * Holds any information related to the object store.\n \t */\n-\tchar *objectdir;\n+\tstruct raw_object_store objects;\n \n \t/*\n \t * Path to the repository's graft file.\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339767","messageId":"20180221015430.96054-3-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 02/27] object-store: migrate alternates struct and functions from cache.h","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:05Z","receivedAt":"2018-02-21T01:54:51Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Migrate the struct alternate_object_database and all its related\nfunctions to the object store as these functions are easier found in\nthat header. The migration is just a verbatim copy, no need to\ninclude the object store header at any C file, because cache.h includes\nrepository.h which in turn includes the object-store.h\n\nSigned-off-by: Stefan Beller \n---\n cache.h | 51 --------------------------------------------------\n object-store.h | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 51 insertions(+), 51 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 9cac7bb518..964873faf2 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1576,57 +1576,6 @@ extern int has_dirs_only_path(const char *name, int len, int prefix_len);\n extern void schedule_dir_for_removal(const char *name, int len);\n extern void remove_scheduled_dirs(void);\n \n-extern struct alternate_object_database {\n-\tstruct alternate_object_database *next;\n-\n-\t/* see alt_scratch_buf() */\n-\tstruct strbuf scratch;\n-\tsize_t base_len;\n-\n-\t/*\n-\t * Used to store the results of readdir(3) calls when searching\n-\t * for unique abbreviated hashes. This cache is never\n-\t * invalidated, thus it's racy and not necessarily accurate.\n-\t * That's fine for its purpose; don't use it for tasks requiring\n-\t * greater accuracy!\n-\t */\n-\tchar loose_objects_subdir_seen[256];\n-\tstruct oid_array loose_objects_cache;\n-\n-\tchar path[FLEX_ARRAY];\n-} *alt_odb_list;\n-extern void prepare_alt_odb(void);\n-extern char *compute_alternate_path(const char *path, struct strbuf *err);\n-typedef int alt_odb_fn(struct alternate_object_database *, void *);\n-extern int foreach_alt_odb(alt_odb_fn, void*);\n-\n-/*\n- * Allocate a \"struct alternate_object_database\" but do _not_ actually\n- * add it to the list of alternates.\n- */\n-struct alternate_object_database *alloc_alt_odb(const char *dir);\n-\n-/*\n- * Add the directory to the on-disk alternates file; the new entry will also\n- * take effect in the current process.\n- */\n-extern void add_to_alternates_file(const char *dir);\n-\n-/*\n- * Add the directory to the in-memory list of alternates (along with any\n- * recursive alternates it points to), but do not modify the on-disk alternates\n- * file.\n- */\n-extern void add_to_alternates_memory(const char *dir);\n-\n-/*\n- * Returns a scratch strbuf pre-filled with the alternate object directory,\n- * including a trailing slash, which can be used to access paths in the\n- * alternate. Always use this over direct access to alt->scratch, as it\n- * cleans up any previous use of the scratch buffer.\n- */\n-extern struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);\n-\n struct pack_window {\n \tstruct pack_window *next;\n \tunsigned char *base;\ndiff --git a/object-store.h b/object-store.h\nindex cf35760ceb..5678aa1136 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -1,6 +1,57 @@\n #ifndef OBJECT_STORE_H\n #define OBJECT_STORE_H\n \n+extern struct alternate_object_database {\n+\tstruct alternate_object_database *next;\n+\n+\t/* see alt_scratch_buf() */\n+\tstruct strbuf scratch;\n+\tsize_t base_len;\n+\n+\t/*\n+\t * Used to store the results of readdir(3) calls when searching\n+\t * for unique abbreviated hashes. This cache is never\n+\t * invalidated, thus it's racy and not necessarily accurate.\n+\t * That's fine for its purpose; don't use it for tasks requiring\n+\t * greater accuracy!\n+\t */\n+\tchar loose_objects_subdir_seen[256];\n+\tstruct oid_array loose_objects_cache;\n+\n+\tchar path[FLEX_ARRAY];\n+} *alt_odb_list;\n+void prepare_alt_odb(void);\n+char *compute_alternate_path(const char *path, struct strbuf *err);\n+typedef int alt_odb_fn(struct alternate_object_database *, void *);\n+int foreach_alt_odb(alt_odb_fn, void*);\n+\n+/*\n+ * Allocate a \"struct alternate_object_database\" but do _not_ actually\n+ * add it to the list of alternates.\n+ */\n+struct alternate_object_database *alloc_alt_odb(const char *dir);\n+\n+/*\n+ * Add the directory to the on-disk alternates file; the new entry will also\n+ * take effect in the current process.\n+ */\n+void add_to_alternates_file(const char *dir);\n+\n+/*\n+ * Add the directory to the in-memory list of alternates (along with any\n+ * recursive alternates it points to), but do not modify the on-disk alternates\n+ * file.\n+ */\n+void add_to_alternates_memory(const char *dir);\n+\n+/*\n+ * Returns a scratch strbuf pre-filled with the alternate object directory,\n+ * including a trailing slash, which can be used to access paths in the\n+ * alternate. Always use this over direct access to alt->scratch, as it\n+ * cleans up any previous use of the scratch buffer.\n+ */\n+struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);\n+\n struct raw_object_store {\n \t/*\n \t * Path to the repository's object store.\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339768","messageId":"20180221015430.96054-6-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 05/27] object-store: move packed_git and packed_git_mru to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:08Z","receivedAt":"2018-02-21T01:54:57Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a process with multiple repositories open, packfile accessors\nshould be associated to a single repository and not shared globally.\nMove packed_git and packed_git_mru into the_repository and adjust\ncallers to reflect this.\n\nPatch generated by\n\n 1. Moving the struct packed_git declaration to object-store.h\n and packed_git, packed_git_mru globals to struct object_store.\n\n 2. Applying the semantic patch\n contrib/coccinelle/refactoring/packed_git.cocci to adjust callers.\n This semantic patch is placed in a sub directory of the coccinelle\n contrib dir, as this semantic patch is not expected to be of general\n usefulness; it is only useful during developing this series and\n merging it with other topics in flight. At a later date, just\n delete that semantic patch.\n\n 3. Applying line wrapping fixes from \"make style\" to break the\n resulting long lines.\n\n 4. Adding missing #includes of repository.h and object-store.h\n where needed.\n\n 5. As the packfiles are now owned by an objectstore/repository, which\n is ephemeral unlike globals, we introduce memory leaks. So address\n them in raw_object_store_clear().\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/count-objects.c | 6 ++++--\n builtin/fsck.c | 7 +++++--\n builtin/gc.c | 4 +++-\n builtin/index-pack.c | 1 +\n builtin/pack-objects.c | 20 +++++++++++---------\n builtin/pack-redundant.c | 6 ++++--\n builtin/receive-pack.c | 1 +\n cache.h | 29 ----------------------------\n fast-import.c | 6 ++++--\n http-backend.c | 6 ++++--\n http-push.c | 1 +\n http-walker.c | 1 +\n http.c | 1 +\n object-store.h | 41 +++++++++++++++++++++++++++++++++++++++-\n object.c | 7 +++++++\n pack-bitmap.c | 4 +++-\n pack-check.c | 1 +\n pack-revindex.c | 1 +\n packfile.c | 39 +++++++++++++++++++-------------------\n reachable.c | 1 +\n repository.c | 2 ++\n server-info.c | 6 ++++--\n sha1_name.c | 5 +++--\n 23 files changed, 122 insertions(+), 74 deletions(-)\n\ndiff --git a/builtin/count-objects.c b/builtin/count-objects.c\nindex 33343818c8..9334648dcc 100644\n--- a/builtin/count-objects.c\n+++ b/builtin/count-objects.c\n@@ -7,6 +7,8 @@\n #include \"cache.h\"\n #include \"config.h\"\n #include \"dir.h\"\n+#include \"repository.h\"\n+#include \"object-store.h\"\n #include \"builtin.h\"\n #include \"parse-options.h\"\n #include \"quote.h\"\n@@ -120,9 +122,9 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)\n \t\tstruct strbuf loose_buf = STRBUF_INIT;\n \t\tstruct strbuf pack_buf = STRBUF_INIT;\n \t\tstruct strbuf garbage_buf = STRBUF_INIT;\n-\t\tif (!packed_git)\n+\t\tif (!the_repository->objects.packed_git)\n \t\t\tprepare_packed_git();\n-\t\tfor (p = packed_git; p; p = p->next) {\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t\tif (!p->pack_local)\n \t\t\t\tcontinue;\n \t\t\tif (open_pack_index(p))\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 908e4f092a..2e99e02128 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -2,6 +2,7 @@\n #include \"cache.h\"\n #include \"repository.h\"\n #include \"config.h\"\n+#include \"object-store.h\"\n #include \"commit.h\"\n #include \"tree.h\"\n #include \"blob.h\"\n@@ -729,7 +730,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t\t\tprepare_packed_git();\n \n \t\t\tif (show_progress) {\n-\t\t\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\t\tfor (p = the_repository->objects.packed_git; p;\n+\t\t\t\t p = p->next) {\n \t\t\t\t\tif (open_pack_index(p))\n \t\t\t\t\t\tcontinue;\n \t\t\t\t\ttotal += p->num_objects;\n@@ -737,7 +739,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \n \t\t\t\tprogress = start_progress(_(\"Checking objects\"), total);\n \t\t\t}\n-\t\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\tfor (p = the_repository->objects.packed_git; p;\n+\t\t\t p = p->next) {\n \t\t\t\t/* verify gives error messages itself */\n \t\t\t\tif (verify_pack(p, fsck_obj_buffer,\n \t\t\t\t\t\tprogress, count))\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 77fa720bd0..96ff790867 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -11,6 +11,7 @@\n */\n \n #include \"builtin.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n #include \"tempfile.h\"\n #include \"lockfile.h\"\n@@ -18,6 +19,7 @@\n #include \"run-command.h\"\n #include \"sigchain.h\"\n #include \"argv-array.h\"\n+#include \"object-store.h\"\n #include \"commit.h\"\n #include \"packfile.h\"\n \n@@ -173,7 +175,7 @@ static int too_many_packs(void)\n \t\treturn 0;\n \n \tprepare_packed_git();\n-\tfor (cnt = 0, p = packed_git; p; p = p->next) {\n+\tfor (cnt = 0, p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (!p->pack_local)\n \t\t\tcontinue;\n \t\tif (p->pack_keep)\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 5ebd370c56..e60cc3ed98 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -12,6 +12,7 @@\n #include \"exec_cmd.h\"\n #include \"streaming.h\"\n #include \"thread-utils.h\"\n+#include \"object-store.h\"\n #include \"packfile.h\"\n \n static const char index_pack_usage[] =\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 83dcbc9773..7f376c2aef 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1,6 +1,8 @@\n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n+#include \"object-store.h\"\n #include \"attr.h\"\n #include \"object.h\"\n #include \"blob.h\"\n@@ -1025,8 +1027,7 @@ static int want_object_in_pack(const struct object_id *oid,\n \t\tif (want != -1)\n \t\t\treturn want;\n \t}\n-\n-\tlist_for_each(pos, &packed_git_mru) {\n+\tlist_for_each(pos, &the_repository->objects.packed_git_mru) {\n \t\tstruct packed_git *p = list_entry(pos, struct packed_git, mru);\n \t\toff_t offset;\n \n@@ -1044,7 +1045,7 @@ static int want_object_in_pack(const struct object_id *oid,\n \t\t\t}\n \t\t\twant = want_found_object(exclude, p);\n \t\t\tif (!exclude && want > 0)\n-\t\t\t\tlist_move(&p->mru, &packed_git_mru);\n+\t\t\t\tlist_move(&p->mru, &the_repository->objects.packed_git_mru);\n \t\t\tif (want != -1)\n \t\t\t\treturn want;\n \t\t}\n@@ -2673,7 +2674,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)\n \n \tmemset(&in_pack, 0, sizeof(in_pack));\n \n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tstruct object_id oid;\n \t\tstruct object *o;\n \n@@ -2736,7 +2737,8 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)\n \tstatic struct packed_git *last_found = (void *)1;\n \tstruct packed_git *p;\n \n-\tp = (last_found != (void *)1) ? last_found : packed_git;\n+\tp = (last_found != (void *)1) ? last_found :\n+\t\t\t\t\tthe_repository->objects.packed_git;\n \n \twhile (p) {\n \t\tif ((!p->pack_local || p->pack_keep) &&\n@@ -2745,7 +2747,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)\n \t\t\treturn 1;\n \t\t}\n \t\tif (p == last_found)\n-\t\t\tp = packed_git;\n+\t\t\tp = the_repository->objects.packed_git;\n \t\telse\n \t\t\tp = p->next;\n \t\tif (p == last_found)\n@@ -2781,7 +2783,7 @@ static void loosen_unused_packed_objects(struct rev_info *revs)\n \tuint32_t i;\n \tstruct object_id oid;\n \n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (!p->pack_local || p->pack_keep)\n \t\t\tcontinue;\n \n@@ -3152,7 +3154,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \tprepare_packed_git();\n \tif (ignore_packed_keep) {\n \t\tstruct packed_git *p;\n-\t\tfor (p = packed_git; p; p = p->next)\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\t\tif (p->pack_local && p->pack_keep)\n \t\t\t\tbreak;\n \t\tif (!p) /* no keep-able packs found */\n@@ -3165,7 +3167,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \t\t * also covers non-local objects\n \t\t */\n \t\tstruct packed_git *p;\n-\t\tfor (p = packed_git; p; p = p->next) {\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t\tif (!p->pack_local) {\n \t\t\t\thave_non_local_packs = 1;\n \t\t\t\tbreak;\ndiff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c\nindex aaa8136322..55462bc826 100644\n--- a/builtin/pack-redundant.c\n+++ b/builtin/pack-redundant.c\n@@ -7,6 +7,8 @@\n */\n \n #include \"builtin.h\"\n+#include \"repository.h\"\n+#include \"object-store.h\"\n #include \"packfile.h\"\n \n #define BLKSIZE 512\n@@ -571,7 +573,7 @@ static struct pack_list * add_pack(struct packed_git *p)\n \n static struct pack_list * add_pack_file(const char *filename)\n {\n-\tstruct packed_git *p = packed_git;\n+\tstruct packed_git *p = the_repository->objects.packed_git;\n \n \tif (strlen(filename) < 40)\n \t\tdie(\"Bad pack filename: %s\", filename);\n@@ -586,7 +588,7 @@ static struct pack_list * add_pack_file(const char *filename)\n \n static void load_all(void)\n {\n-\tstruct packed_git *p = packed_git;\n+\tstruct packed_git *p = the_repository->objects.packed_git;\n \n \twhile (p) {\n \t\tadd_pack(p);\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex b7ce7c7f52..b2eac79a6e 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -7,6 +7,7 @@\n #include \"sideband.h\"\n #include \"run-command.h\"\n #include \"exec_cmd.h\"\n+#include \"object-store.h\"\n #include \"commit.h\"\n #include \"object.h\"\n #include \"remote.h\"\ndiff --git a/cache.h b/cache.h\nindex 964873faf2..3fd9dc52cd 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1585,35 +1585,6 @@ struct pack_window {\n \tunsigned int inuse_cnt;\n };\n \n-extern struct packed_git {\n-\tstruct packed_git *next;\n-\tstruct list_head mru;\n-\tstruct pack_window *windows;\n-\toff_t pack_size;\n-\tconst void *index_data;\n-\tsize_t index_size;\n-\tuint32_t num_objects;\n-\tuint32_t num_bad_objects;\n-\tunsigned char *bad_object_sha1;\n-\tint index_version;\n-\ttime_t mtime;\n-\tint pack_fd;\n-\tunsigned pack_local:1,\n-\t\t pack_keep:1,\n-\t\t freshened:1,\n-\t\t do_not_close:1,\n-\t\t pack_promisor:1;\n-\tunsigned char sha1[20];\n-\tstruct revindex_entry *revindex;\n-\t/* something like \".git/objects/pack/xxxxx.pack\" */\n-\tchar pack_name[FLEX_ARRAY]; /* more */\n-} *packed_git;\n-\n-/*\n- * A most-recently-used ordered version of the packed_git list.\n- */\n-extern struct list_head packed_git_mru;\n-\n struct pack_entry {\n \toff_t offset;\n \tunsigned char sha1[20];\ndiff --git a/fast-import.c b/fast-import.c\nindex b70ac025e0..1685fe59a2 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -154,8 +154,10 @@ Format of STDIN stream:\n \n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n #include \"lockfile.h\"\n+#include \"object-store.h\"\n #include \"object.h\"\n #include \"blob.h\"\n #include \"tree.h\"\n@@ -1110,7 +1112,7 @@ static int store_object(\n \tif (e->idx.offset) {\n \t\tduplicate_count_by_type[type]++;\n \t\treturn 1;\n-\t} else if (find_sha1_pack(oid.hash, packed_git)) {\n+\t} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {\n \t\te->type = type;\n \t\te->pack_id = MAX_PACK_ID;\n \t\te->idx.offset = 1; /* just not zero! */\n@@ -1305,7 +1307,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)\n \t\tduplicate_count_by_type[OBJ_BLOB]++;\n \t\ttruncate_pack(&checkpoint);\n \n-\t} else if (find_sha1_pack(oid.hash, packed_git)) {\n+\t} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {\n \t\te->type = OBJ_BLOB;\n \t\te->pack_id = MAX_PACK_ID;\n \t\te->idx.offset = 1; /* just not zero! */\ndiff --git a/http-backend.c b/http-backend.c\nindex f3dc218b2a..4d93126c15 100644\n--- a/http-backend.c\n+++ b/http-backend.c\n@@ -1,7 +1,9 @@\n #include \"cache.h\"\n #include \"config.h\"\n+#include \"repository.h\"\n #include \"refs.h\"\n #include \"pkt-line.h\"\n+#include \"object-store.h\"\n #include \"object.h\"\n #include \"tag.h\"\n #include \"exec_cmd.h\"\n@@ -518,13 +520,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg)\n \n \tselect_getanyfile(hdr);\n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (p->pack_local)\n \t\t\tcnt++;\n \t}\n \n \tstrbuf_grow(&buf, cnt * 53 + 2);\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (p->pack_local)\n \t\t\tstrbuf_addf(&buf, \"P %s\\n\", p->pack_name + objdirlen + 6);\n \t}\ndiff --git a/http-push.c b/http-push.c\nindex 0913f8ab86..1ca662390f 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -1,4 +1,5 @@\n #include \"cache.h\"\n+#include \"object-store.h\"\n #include \"commit.h\"\n #include \"tag.h\"\n #include \"blob.h\"\ndiff --git a/http-walker.c b/http-walker.c\nindex 07c2b1af82..8bb5d991bb 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -4,6 +4,7 @@\n #include \"http.h\"\n #include \"list.h\"\n #include \"transport.h\"\n+#include \"object-store.h\"\n #include \"packfile.h\"\n \n struct alt_base {\ndiff --git a/http.c b/http.c\nindex 31755023a4..a4a9e583c7 100644\n--- a/http.c\n+++ b/http.c\n@@ -1,6 +1,7 @@\n #include \"git-compat-util.h\"\n #include \"http.h\"\n #include \"config.h\"\n+#include \"object-store.h\"\n #include \"pack.h\"\n #include \"sideband.h\"\n #include \"run-command.h\"\ndiff --git a/object-store.h b/object-store.h\nindex e78eea1dde..1de9e07102 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -52,6 +52,30 @@ void add_to_alternates_memory(const char *dir);\n */\n struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);\n \n+struct packed_git {\n+\tstruct packed_git *next;\n+\tstruct list_head mru;\n+\tstruct pack_window *windows;\n+\toff_t pack_size;\n+\tconst void *index_data;\n+\tsize_t index_size;\n+\tuint32_t num_objects;\n+\tuint32_t num_bad_objects;\n+\tunsigned char *bad_object_sha1;\n+\tint index_version;\n+\ttime_t mtime;\n+\tint pack_fd;\n+\tunsigned pack_local:1,\n+\t\t pack_keep:1,\n+\t\t freshened:1,\n+\t\t do_not_close:1,\n+\t\t pack_promisor:1;\n+\tunsigned char sha1[20];\n+\tstruct revindex_entry *revindex;\n+\t/* something like \".git/objects/pack/xxxxx.pack\" */\n+\tchar pack_name[FLEX_ARRAY]; /* more */\n+};\n+\n struct raw_object_store {\n \t/*\n \t * Path to the repository's object store.\n@@ -59,10 +83,25 @@ struct raw_object_store {\n \t */\n \tchar *objectdir;\n \n+\tstruct packed_git *packed_git;\n+\t/*\n+\t * A most-recently-used ordered version of the packed_git list, which can\n+\t * be iterated instead of packed_git (and marked via mru_mark).\n+\t */\n+\tstruct list_head packed_git_mru;\n+\n \tstruct alternate_object_database *alt_odb_list;\n \tstruct alternate_object_database **alt_odb_tail;\n };\n-#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }\n+\n+/*\n+ * The mru list_head is supposed to be initialized using\n+ * the LIST_HEAD macro, assigning prev/next to itself.\n+ * However this doesn't work in this case as some compilers dislike\n+ * that macro on member variables. Use NULL instead as that is defined\n+ * and accepted, deferring the real init to prepare_packed_git_mru(). */\n+#define __MRU_INIT { NULL, NULL }\n+#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/object.c b/object.c\nindex 17b1da6d6b..c76b62572a 100644\n--- a/object.c\n+++ b/object.c\n@@ -466,4 +466,11 @@ void raw_object_store_clear(struct raw_object_store *o)\n \n \tfree_alt_odbs(o);\n \to->alt_odb_tail = NULL;\n+\n+\twhile (!list_empty(&o->packed_git_mru))\n+\t\tlist_del(&o->packed_git_mru);\n+\t/*\n+\t * TODO: call close_all_packs once migrated to\n+\t * take an object store argument\n+\t */\n }\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9270983e5f..d51172b1d5 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -5,11 +5,13 @@\n #include \"revision.h\"\n #include \"progress.h\"\n #include \"list-objects.h\"\n+#include \"object-store.h\"\n #include \"pack.h\"\n #include \"pack-bitmap.h\"\n #include \"pack-revindex.h\"\n #include \"pack-objects.h\"\n #include \"packfile.h\"\n+#include \"repository.h\"\n \n /*\n * An entry on the bitmap index, representing the bitmap for a given\n@@ -335,7 +337,7 @@ static int open_pack_bitmap(void)\n \tassert(!bitmap_git.map && !bitmap_git.loaded);\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (open_pack_bitmap_1(p) == 0)\n \t\t\tret = 0;\n \t}\ndiff --git a/pack-check.c b/pack-check.c\nindex 073c1fbd46..c80c3ed8fd 100644\n--- a/pack-check.c\n+++ b/pack-check.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"pack.h\"\n+#include \"object-store.h\"\n #include \"pack-revindex.h\"\n #include \"progress.h\"\n #include \"packfile.h\"\ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex ff5f62c033..bb521cf7fb 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"pack-revindex.h\"\n+#include \"object-store.h\"\n \n /*\n * Pack index for existing packs give us easy access to the offsets into\ndiff --git a/packfile.c b/packfile.c\nindex 216ea836ee..d41e4c83d0 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -7,6 +7,7 @@\n #include \"packfile.h\"\n #include \"delta.h\"\n #include \"list.h\"\n+#include \"object-store.h\"\n #include \"streaming.h\"\n #include \"sha1-lookup.h\"\n #include \"commit.h\"\n@@ -45,8 +46,6 @@ static unsigned int pack_open_fds;\n static unsigned int pack_max_fds;\n static size_t peak_pack_mapped;\n static size_t pack_mapped;\n-struct packed_git *packed_git;\n-LIST_HEAD(packed_git_mru);\n \n #define SZ_FMT PRIuMAX\n static inline uintmax_t sz_fmt(size_t s) { return s; }\n@@ -246,7 +245,7 @@ static int unuse_one_window(struct packed_git *current)\n \n \tif (current)\n \t\tscan_windows(current, &lru_p, &lru_w, &lru_l);\n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tscan_windows(p, &lru_p, &lru_w, &lru_l);\n \tif (lru_p) {\n \t\tmunmap(lru_w->base, lru_w->len);\n@@ -316,7 +315,7 @@ void close_all_packs(void)\n {\n \tstruct packed_git *p;\n \n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tif (p->do_not_close)\n \t\t\tdie(\"BUG: want to close pack marked 'do-not-close'\");\n \t\telse\n@@ -384,7 +383,7 @@ static int close_one_pack(void)\n \tstruct pack_window *mru_w = NULL;\n \tint accept_windows_inuse = 1;\n \n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (p->pack_fd == -1)\n \t\t\tcontinue;\n \t\tfind_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);\n@@ -686,8 +685,8 @@ void install_packed_git(struct packed_git *pack)\n \tif (pack->pack_fd != -1)\n \t\tpack_open_fds++;\n \n-\tpack->next = packed_git;\n-\tpacked_git = pack;\n+\tpack->next = the_repository->objects.packed_git;\n+\tthe_repository->objects.packed_git = pack;\n }\n \n void (*report_garbage)(unsigned seen_bits, const char *path);\n@@ -769,7 +768,8 @@ static void prepare_packed_git_one(char *objdir, int local)\n \t\tbase_len = path.len;\n \t\tif (strip_suffix_mem(path.buf, &base_len, \".idx\")) {\n \t\t\t/* Don't reopen a pack we already have. */\n-\t\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\tfor (p = the_repository->objects.packed_git; p;\n+\t\t\t p = p->next) {\n \t\t\t\tsize_t len;\n \t\t\t\tif (strip_suffix(p->pack_name, \".pack\", &len) &&\n \t\t\t\t len == base_len &&\n@@ -820,7 +820,7 @@ unsigned long approximate_object_count(void)\n \n \t\tprepare_packed_git();\n \t\tcount = 0;\n-\t\tfor (p = packed_git; p; p = p->next) {\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t\tif (open_pack_index(p))\n \t\t\t\tcontinue;\n \t\t\tcount += p->num_objects;\n@@ -869,18 +869,19 @@ static int sort_pack(const void *a_, const void *b_)\n \n static void rearrange_packed_git(void)\n {\n-\tpacked_git = llist_mergesort(packed_git, get_next_packed_git,\n-\t\t\t\t set_next_packed_git, sort_pack);\n+\tthe_repository->objects.packed_git = llist_mergesort(\n+\t\tthe_repository->objects.packed_git, get_next_packed_git,\n+\t\tset_next_packed_git, sort_pack);\n }\n \n static void prepare_packed_git_mru(void)\n {\n \tstruct packed_git *p;\n \n-\tINIT_LIST_HEAD(&packed_git_mru);\n+\tINIT_LIST_HEAD(&the_repository->objects.packed_git_mru);\n \n-\tfor (p = packed_git; p; p = p->next)\n-\t\tlist_add_tail(&p->mru, &packed_git_mru);\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n+\t\tlist_add_tail(&p->mru, &the_repository->objects.packed_git_mru);\n }\n \n static int prepare_packed_git_run_once = 0;\n@@ -1014,7 +1015,7 @@ const struct packed_git *has_packed_and_bad(const unsigned char *sha1)\n \tstruct packed_git *p;\n \tunsigned i;\n \n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tfor (i = 0; i < p->num_bad_objects; i++)\n \t\t\tif (!hashcmp(sha1, p->bad_object_sha1 + 20 * i))\n \t\t\t\treturn p;\n@@ -1845,13 +1846,13 @@ int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)\n \tstruct list_head *pos;\n \n \tprepare_packed_git();\n-\tif (!packed_git)\n+\tif (!the_repository->objects.packed_git)\n \t\treturn 0;\n \n-\tlist_for_each(pos, &packed_git_mru) {\n+\tlist_for_each(pos, &the_repository->objects.packed_git_mru) {\n \t\tstruct packed_git *p = list_entry(pos, struct packed_git, mru);\n \t\tif (fill_pack_entry(sha1, e, p)) {\n-\t\t\tlist_move(&p->mru, &packed_git_mru);\n+\t\t\tlist_move(&p->mru, &the_repository->objects.packed_git_mru);\n \t\t\treturn 1;\n \t\t}\n \t}\n@@ -1898,7 +1899,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)\n \tint pack_errors = 0;\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)\n \t\t\tcontinue;\n \t\tif ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&\ndiff --git a/reachable.c b/reachable.c\nindex 88d7d679da..59ed16ba76 100644\n--- a/reachable.c\n+++ b/reachable.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"refs.h\"\n+#include \"object-store.h\"\n #include \"tag.h\"\n #include \"commit.h\"\n #include \"blob.h\"\ndiff --git a/repository.c b/repository.c\nindex 2255ff657e..9798f4ffca 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -142,6 +142,8 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)\n \n \trepo->ignore_env = 1;\n \n+\tINIT_LIST_HEAD(&repo->objects.packed_git_mru);\n+\n \tif (repo_init_gitdir(repo, gitdir))\n \t\tgoto error;\n \ndiff --git a/server-info.c b/server-info.c\nindex 26a6c20b7d..96885c45f6 100644\n--- a/server-info.c\n+++ b/server-info.c\n@@ -1,5 +1,7 @@\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"refs.h\"\n+#include \"object-store.h\"\n #include \"object.h\"\n #include \"commit.h\"\n #include \"tag.h\"\n@@ -200,7 +202,7 @@ static void init_pack_info(const char *infofile, int force)\n \tobjdirlen = strlen(objdir);\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t/* we ignore things on alternate path since they are\n \t\t * not available to the pullers in general.\n \t\t */\n@@ -210,7 +212,7 @@ static void init_pack_info(const char *infofile, int force)\n \t}\n \tnum_pack = i;\n \tinfo = xcalloc(num_pack, sizeof(struct pack_info *));\n-\tfor (i = 0, p = packed_git; p; p = p->next) {\n+\tfor (i = 0, p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (!p->pack_local)\n \t\t\tcontinue;\n \t\tinfo[i] = xcalloc(1, sizeof(struct pack_info));\ndiff --git a/sha1_name.c b/sha1_name.c\nindex 957ce25680..016c883b5c 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -196,7 +196,8 @@ static void find_short_packed_object(struct disambiguate_state *ds)\n \tstruct packed_git *p;\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p && !ds->ambiguous; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p && !ds->ambiguous;\n+\t p = p->next)\n \t\tunique_in_pack(p, ds);\n }\n \n@@ -566,7 +567,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)\n \tstruct packed_git *p;\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tfind_abbrev_len_for_pack(p, mad);\n }\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339769","messageId":"20180221015430.96054-9-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 08/27] pack: move approximate object count to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:11Z","receivedAt":"2018-02-21T01:54:59Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The approximate_object_count() function maintains a rough count of\nobjects in a repository to estimate how long object name abbreviates\nshould be. Object names are scoped to a repository and the\nappropriate length may differ by repository, so the object count\nshould not be global.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 10 +++++++++-\n packfile.c | 11 +++++------\n 2 files changed, 14 insertions(+), 7 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 6cecba3951..bd1e4fcd8b 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -93,6 +93,14 @@ struct raw_object_store {\n \tstruct alternate_object_database *alt_odb_list;\n \tstruct alternate_object_database **alt_odb_tail;\n \n+\t/*\n+\t * A fast, rough count of the number of objects in the repository.\n+\t * These two fields are not meant for direct access. Use\n+\t * approximate_object_count() instead.\n+\t */\n+\tunsigned long approximate_object_count;\n+\tunsigned approximate_object_count_valid : 1;\n+\n \t/*\n \t * Whether packed_git has already been populated with this repository's\n \t * packs.\n@@ -107,7 +115,7 @@ struct raw_object_store {\n * that macro on member variables. Use NULL instead as that is defined\n * and accepted, deferring the real init to prepare_packed_git_mru(). */\n #define __MRU_INIT { NULL, NULL }\n-#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0 }\n+#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0, 0, 0 }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/packfile.c b/packfile.c\nindex a8a2e7fe43..693bafbc98 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -803,8 +803,6 @@ static void prepare_packed_git_one(char *objdir, int local)\n \tstrbuf_release(&path);\n }\n \n-static int approximate_object_count_valid;\n-\n /*\n * Give a fast, rough count of the number of objects in the repository. This\n * ignores loose objects completely. If you have a lot of them, then either\n@@ -814,8 +812,8 @@ static int approximate_object_count_valid;\n */\n unsigned long approximate_object_count(void)\n {\n-\tstatic unsigned long count;\n-\tif (!approximate_object_count_valid) {\n+\tif (!the_repository->objects.approximate_object_count_valid) {\n+\t\tunsigned long count;\n \t\tstruct packed_git *p;\n \n \t\tprepare_packed_git();\n@@ -825,8 +823,9 @@ unsigned long approximate_object_count(void)\n \t\t\t\tcontinue;\n \t\t\tcount += p->num_objects;\n \t\t}\n+\t\tthe_repository->objects.approximate_object_count = count;\n \t}\n-\treturn count;\n+\treturn the_repository->objects.approximate_object_count;\n }\n \n static void *get_next_packed_git(const void *p)\n@@ -901,7 +900,7 @@ void prepare_packed_git(void)\n \n void reprepare_packed_git(void)\n {\n-\tapproximate_object_count_valid = 0;\n+\tthe_repository->objects.approximate_object_count_valid = 0;\n \tthe_repository->objects.packed_git_initialized = 0;\n \tprepare_packed_git();\n }\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339770","messageId":"20180221015430.96054-11-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 10/27] sha1_file: add repository argument to link_alt_odb_entry","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:13Z","receivedAt":"2018-02-21T01:55:00Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the link_alt_odb_entry caller to be\nmore specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nSince the implementation does not yet work with other repositories,\nuse a wrapper macro to enforce that the caller passes in\nthe_repository as the first argument. It would be more appealing to\nuse BUILD_ASSERT_OR_ZERO to enforce this, but that doesn't work\nbecause it requires a compile-time constant and common compilers like\ngcc 4.8.4 do not consider \"r == the_repository\" a compile-time\nconstant.\n\nThis and the following three patches add repository arguments to\nlink_alt_odb_entry, read_info_alternates, link_alt_odb_entries\nand prepare_alt_odb. Three out of the four functions are found\nin a recursive call chain, calling each other, and one of them\naccesses the repositories `objectdir` (which was migrated; it\nwas an obvious choice) and `ignore_env` (which we need to keep in\nthe repository struct for clarify); hence we will pass through the\nrepository unlike just the object store object + the ignore_env flag.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex a8e23bd2f8..aaa6276211 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -390,8 +390,9 @@ static int alt_odb_usable(struct raw_object_store *o,\n * terminating NUL.\n */\n static void read_info_alternates(const char * relative_base, int depth);\n-static int link_alt_odb_entry(const char *entry, const char *relative_base,\n-\tint depth, const char *normalized_objdir)\n+#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)\n+static int link_alt_odb_entry_the_repository(const char *entry,\n+\tconst char *relative_base, int depth, const char *normalized_objdir)\n {\n \tstruct alternate_object_database *ent;\n \tstruct strbuf pathbuf = STRBUF_INIT;\n@@ -488,7 +489,8 @@ static void link_alt_odb_entries(const char *alt, int sep,\n \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n \t\tif (!entry.len)\n \t\t\tcontinue;\n-\t\tlink_alt_odb_entry(entry.buf, relative_base, depth, objdirbuf.buf);\n+\t\tlink_alt_odb_entry(the_repository, entry.buf,\n+\t\t\t\t relative_base, depth, objdirbuf.buf);\n \t}\n \tstrbuf_release(&entry);\n \tstrbuf_release(&objdirbuf);\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339771","messageId":"20180221015430.96054-12-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 11/27] sha1_file: add repository argument to read_info_alternates","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:14Z","receivedAt":"2018-02-21T01:55:04Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"See previous patch for explanation.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 +++++++----\n 1 file changed, 7 insertions(+), 4 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex aaa6276211..6edcc3d3a7 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -389,7 +389,9 @@ static int alt_odb_usable(struct raw_object_store *o,\n * SHA1, an extra slash for the first level indirection, and the\n * terminating NUL.\n */\n-static void read_info_alternates(const char * relative_base, int depth);\n+#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)\n+static void read_info_alternates_the_repository(const char *relative_base,\n+\t\t\t\t\t\tint depth);\n #define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)\n static int link_alt_odb_entry_the_repository(const char *entry,\n \tconst char *relative_base, int depth, const char *normalized_objdir)\n@@ -430,7 +432,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,\n \tent->next = NULL;\n \n \t/* recursively add alternates */\n-\tread_info_alternates(pathbuf.buf, depth + 1);\n+\tread_info_alternates(the_repository, pathbuf.buf, depth + 1);\n \n \tstrbuf_release(&pathbuf);\n \treturn 0;\n@@ -496,7 +498,8 @@ static void link_alt_odb_entries(const char *alt, int sep,\n \tstrbuf_release(&objdirbuf);\n }\n \n-static void read_info_alternates(const char * relative_base, int depth)\n+static void read_info_alternates_the_repository(const char *relative_base,\n+\t\t\t\t\t\tint depth)\n {\n \tchar *path;\n \tstruct strbuf buf = STRBUF_INIT;\n@@ -680,7 +683,7 @@ void prepare_alt_odb(void)\n \t\t\t&the_repository->objects.alt_odb_list;\n \tlink_alt_odb_entries(alt, PATH_SEP, NULL, 0);\n \n-\tread_info_alternates(get_object_directory(), 0);\n+\tread_info_alternates(the_repository, get_object_directory(), 0);\n }\n \n /* Returns 1 if we have successfully freshened the file, 0 otherwise. */\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339772","messageId":"20180221015430.96054-14-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 13/27] sha1_file: add repository argument to prepare_alt_odb","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:16Z","receivedAt":"2018-02-21T01:55:07Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"See previous patch for explanation.\n\nWhile at it, move the declaration to object-store.h,\nwhere it should be easier to find.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/fsck.c | 2 +-\n object-store.h | 3 ++-\n packfile.c | 2 +-\n sha1_file.c | 13 +++++++------\n sha1_name.c | 3 ++-\n 5 files changed, 13 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 2e99e02128..76c94e9b5a 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -717,7 +717,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t} else {\n \t\tfsck_object_dir(get_object_directory());\n \n-\t\tprepare_alt_odb();\n+\t\tprepare_alt_odb(the_repository);\n \t\tfor (alt = the_repository->objects.alt_odb_list;\n \t\t\t\talt; alt = alt->next)\n \t\t\tfsck_object_dir(alt->path);\ndiff --git a/object-store.h b/object-store.h\nindex bd1e4fcd8b..514ad94287 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -20,7 +20,8 @@ struct alternate_object_database {\n \n \tchar path[FLEX_ARRAY];\n };\n-void prepare_alt_odb(void);\n+#define prepare_alt_odb(r) prepare_alt_odb_##r()\n+void prepare_alt_odb_the_repository(void);\n char *compute_alternate_path(const char *path, struct strbuf *err);\n typedef int alt_odb_fn(struct alternate_object_database *, void *);\n int foreach_alt_odb(alt_odb_fn, void*);\ndiff --git a/packfile.c b/packfile.c\nindex 693bafbc98..ccfe084642 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -890,7 +890,7 @@ void prepare_packed_git(void)\n \tif (the_repository->objects.packed_git_initialized)\n \t\treturn;\n \tprepare_packed_git_one(get_object_directory(), 1);\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)\n \t\tprepare_packed_git_one(alt->path, 0);\n \trearrange_packed_git();\ndiff --git a/sha1_file.c b/sha1_file.c\nindex a0e9116318..c26921df4a 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -23,6 +23,7 @@\n #include \"sha1-lookup.h\"\n #include \"bulk-checkin.h\"\n #include \"repository.h\"\n+#include \"object-store.h\"\n #include \"streaming.h\"\n #include \"dir.h\"\n #include \"list.h\"\n@@ -581,7 +582,7 @@ void add_to_alternates_memory(const char *reference)\n \t * Make sure alternates are initialized, or else our entry may be\n \t * overwritten when they are.\n \t */\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \n \tlink_alt_odb_entries(the_repository, reference,\n \t\t\t '\\n', NULL, 0);\n@@ -667,7 +668,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \tstruct alternate_object_database *ent;\n \tint r = 0;\n \n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \tfor (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {\n \t\tr = fn(ent, cb);\n \t\tif (r)\n@@ -676,7 +677,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \treturn r;\n }\n \n-void prepare_alt_odb(void)\n+void prepare_alt_odb_the_repository(void)\n {\n \tconst char *alt;\n \n@@ -730,7 +731,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)\n static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)\n {\n \tstruct alternate_object_database *alt;\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\tconst char *path = alt_sha1_path(alt, sha1);\n \t\tif (check_and_freshen_file(path, freshen))\n@@ -889,7 +890,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n \tif (!lstat(*path, st))\n \t\treturn 0;\n \n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \terrno = ENOENT;\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n@@ -920,7 +921,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \t\treturn fd;\n \tmost_interesting_errno = errno;\n \n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tfd = git_open(*path);\ndiff --git a/sha1_name.c b/sha1_name.c\nindex 016c883b5c..3e490ee8f6 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -11,6 +11,7 @@\n #include \"sha1-array.h\"\n #include \"packfile.h\"\n #include \"repository.h\"\n+#include \"object-store.h\"\n \n static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);\n \n@@ -353,7 +354,7 @@ static int init_object_disambiguation(const char *name, int len,\n \n \tds->len = len;\n \tds->hex_pfx[len] = '\\0';\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \treturn 0;\n }\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339773","messageId":"20180221015430.96054-16-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 15/27] sha1_file: allow prepare_alt_odb to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:18Z","receivedAt":"2018-02-21T01:55:11Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n object-store.h | 3 +--\n sha1_file.c | 21 +++++++++++----------\n 2 files changed, 12 insertions(+), 12 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex f283fbdba9..873b341774 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -24,8 +24,7 @@ struct alternate_object_database {\n \t */\n \tchar path[FLEX_ARRAY];\n };\n-#define prepare_alt_odb(r) prepare_alt_odb_##r()\n-void prepare_alt_odb_the_repository(void);\n+void prepare_alt_odb(struct repository *r);\n char *compute_alternate_path(const char *path, struct strbuf *err);\n typedef int alt_odb_fn(struct alternate_object_database *, void *);\n int foreach_alt_odb(alt_odb_fn, void*);\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 6e5105a252..9cf3fffbeb 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -673,21 +673,22 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \treturn r;\n }\n \n-void prepare_alt_odb_the_repository(void)\n+void prepare_alt_odb(struct repository *r)\n {\n-\tconst char *alt;\n-\n-\tif (the_repository->objects.alt_odb_tail)\n+\tif (r->objects.alt_odb_tail)\n \t\treturn;\n \n-\talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n+\tr->objects.alt_odb_tail = &r->objects.alt_odb_list;\n+\n+\tif (!r->ignore_env) {\n+\t\tconst char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);\n+\t\tif (!alt)\n+\t\t\talt = \"\";\n \n-\tthe_repository->objects.alt_odb_tail =\n-\t\t\t&the_repository->objects.alt_odb_list;\n-\tlink_alt_odb_entries(the_repository, alt,\n-\t\t\t PATH_SEP, NULL, 0);\n+\t\tlink_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);\n+\t}\n \n-\tread_info_alternates(the_repository, get_object_directory(), 0);\n+\tread_info_alternates(r, r->objects.objectdir, 0);\n }\n \n /* Returns 1 if we have successfully freshened the file, 0 otherwise. */\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339774","messageId":"20180221015430.96054-15-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 14/27] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:17Z","receivedAt":"2018-02-21T01:55:12Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Actually this also allows read_info_alternates and link_alt_odb_entry to\nhandle arbitrary repositories, but link_alt_odb_entries is the most\ninteresting function in this set of functions, hence the commit subject.\n\nThese functions span a strongly connected component in the function\ngraph, i.e. the recursive call chain might look like\n\n -> link_alt_odb_entries\n -> link_alt_odb_entry\n -> read_info_alternates\n -> link_alt_odb_entries\n\nThat is why we need to convert all these functions at the same time.\n\nSigned-off-by: Jonathan Nieder \nSigned-off-by: Stefan Beller \n---\n object-store.h | 4 ++++\n sha1_file.c | 36 ++++++++++++++++--------------------\n 2 files changed, 20 insertions(+), 20 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 514ad94287..f283fbdba9 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -18,6 +18,10 @@ struct alternate_object_database {\n \tchar loose_objects_subdir_seen[256];\n \tstruct oid_array loose_objects_cache;\n \n+\t/*\n+\t * Path to the alternative object store. If this is a relative path,\n+\t * it is relative to the current working directory.\n+\t */\n \tchar path[FLEX_ARRAY];\n };\n #define prepare_alt_odb(r) prepare_alt_odb_##r()\ndiff --git a/sha1_file.c b/sha1_file.c\nindex c26921df4a..6e5105a252 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -390,11 +390,10 @@ static int alt_odb_usable(struct raw_object_store *o,\n * SHA1, an extra slash for the first level indirection, and the\n * terminating NUL.\n */\n-#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)\n-static void read_info_alternates_the_repository(const char *relative_base,\n-\t\t\t\t\t\tint depth);\n-#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)\n-static int link_alt_odb_entry_the_repository(const char *entry,\n+static void read_info_alternates(struct repository *r,\n+\t\t\t\t const char *relative_base,\n+\t\t\t\t int depth);\n+static int link_alt_odb_entry(struct repository *r, const char *entry,\n \tconst char *relative_base, int depth, const char *normalized_objdir)\n {\n \tstruct alternate_object_database *ent;\n@@ -420,7 +419,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,\n \twhile (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')\n \t\tstrbuf_setlen(&pathbuf, pathbuf.len - 1);\n \n-\tif (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {\n+\tif (!alt_odb_usable(&r->objects, &pathbuf, normalized_objdir)) {\n \t\tstrbuf_release(&pathbuf);\n \t\treturn -1;\n \t}\n@@ -428,12 +427,12 @@ static int link_alt_odb_entry_the_repository(const char *entry,\n \tent = alloc_alt_odb(pathbuf.buf);\n \n \t/* add the alternate entry */\n-\t*the_repository->objects.alt_odb_tail = ent;\n-\tthe_repository->objects.alt_odb_tail = &(ent->next);\n+\t*r->objects.alt_odb_tail = ent;\n+\tr->objects.alt_odb_tail = &(ent->next);\n \tent->next = NULL;\n \n \t/* recursively add alternates */\n-\tread_info_alternates(the_repository, pathbuf.buf, depth + 1);\n+\tread_info_alternates(r, pathbuf.buf, depth + 1);\n \n \tstrbuf_release(&pathbuf);\n \treturn 0;\n@@ -468,12 +467,8 @@ static const char *parse_alt_odb_entry(const char *string,\n \treturn end;\n }\n \n-#define link_alt_odb_entries(r, a, s, rb, d) \\\n-\tlink_alt_odb_entries_##r(a, s, rb, d)\n-static void link_alt_odb_entries_the_repository(const char *alt,\n-\t\t\t\t\t\tint sep,\n-\t\t\t\t\t\tconst char *relative_base,\n-\t\t\t\t\t\tint depth)\n+static void link_alt_odb_entries(struct repository *r, const char *alt,\n+\t\t\t\t int sep, const char *relative_base, int depth)\n {\n \tstruct strbuf objdirbuf = STRBUF_INIT;\n \tstruct strbuf entry = STRBUF_INIT;\n@@ -487,7 +482,7 @@ static void link_alt_odb_entries_the_repository(const char *alt,\n \t\treturn;\n \t}\n \n-\tstrbuf_add_absolute_path(&objdirbuf, get_object_directory());\n+\tstrbuf_add_absolute_path(&objdirbuf, r->objects.objectdir);\n \tif (strbuf_normalize_path(&objdirbuf) < 0)\n \t\tdie(\"unable to normalize object directory: %s\",\n \t\t objdirbuf.buf);\n@@ -496,15 +491,16 @@ static void link_alt_odb_entries_the_repository(const char *alt,\n \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n \t\tif (!entry.len)\n \t\t\tcontinue;\n-\t\tlink_alt_odb_entry(the_repository, entry.buf,\n+\t\tlink_alt_odb_entry(r, entry.buf,\n \t\t\t\t relative_base, depth, objdirbuf.buf);\n \t}\n \tstrbuf_release(&entry);\n \tstrbuf_release(&objdirbuf);\n }\n \n-static void read_info_alternates_the_repository(const char *relative_base,\n-\t\t\t\t\t\tint depth)\n+static void read_info_alternates(struct repository *r,\n+\t\t\t\t const char *relative_base,\n+\t\t\t\t int depth)\n {\n \tchar *path;\n \tstruct strbuf buf = STRBUF_INIT;\n@@ -516,7 +512,7 @@ static void read_info_alternates_the_repository(const char *relative_base,\n \t\treturn;\n \t}\n \n-\tlink_alt_odb_entries(the_repository, buf.buf, '\\n', relative_base, depth);\n+\tlink_alt_odb_entries(r, buf.buf, '\\n', relative_base, depth);\n \tstrbuf_release(&buf);\n \tfree(path);\n }\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339775","messageId":"20180221015430.96054-10-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 09/27] sha1_file: add raw_object_store argument to alt_odb_usable","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:12Z","receivedAt":"2018-02-21T01:55:14Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a raw_object_store to alt_odb_usable to be more specific about which\nrepository to act on. The choice of the repository is delegated to its\nonly caller link_alt_odb_entry.\n\nSigned-off-by: Stefan Beller \n---\n sha1_file.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 1348dce68f..a8e23bd2f8 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -346,7 +346,9 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\n-static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)\n+static int alt_odb_usable(struct raw_object_store *o,\n+\t\t\t struct strbuf *path,\n+\t\t\t const char *normalized_objdir)\n {\n \tstruct alternate_object_database *alt;\n \n@@ -362,7 +364,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)\n \t * Prevent the common mistake of listing the same\n \t * thing twice, or object directory itself.\n \t */\n-\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = o->alt_odb_list; alt; alt = alt->next) {\n \t\tif (!fspathcmp(path->buf, alt->path))\n \t\t\treturn 0;\n \t}\n@@ -414,7 +416,7 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,\n \twhile (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')\n \t\tstrbuf_setlen(&pathbuf, pathbuf.len - 1);\n \n-\tif (!alt_odb_usable(&pathbuf, normalized_objdir)) {\n+\tif (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {\n \t\tstrbuf_release(&pathbuf);\n \t\treturn -1;\n \t}\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339776","messageId":"20180221015430.96054-20-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 19/27] sha1_file: add repository argument to map_sha1_file_1","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:22Z","receivedAt":"2018-02-21T01:55:20Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the map_sha1_file_1 caller to be\nmore specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 ++++++-----\n 1 file changed, 6 insertions(+), 5 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 68aff90792..7fa9ed2f25 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -938,9 +938,10 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,\n * Map the loose object at \"path\" if it is not NULL, or the path found by\n * searching for a loose object named \"sha1\".\n */\n-static void *map_sha1_file_1(const char *path,\n-\t\t\t const unsigned char *sha1,\n-\t\t\t unsigned long *size)\n+#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)\n+static void *map_sha1_file_1_the_repository(const char *path,\n+\t\t\t\t\t const unsigned char *sha1,\n+\t\t\t\t\t unsigned long *size)\n {\n \tvoid *map;\n \tint fd;\n@@ -969,7 +970,7 @@ static void *map_sha1_file_1(const char *path,\n \n void *map_sha1_file(const unsigned char *sha1, unsigned long *size)\n {\n-\treturn map_sha1_file_1(NULL, sha1, size);\n+\treturn map_sha1_file_1(the_repository, NULL, sha1, size);\n }\n \n static int unpack_sha1_short_header(git_zstream *stream,\n@@ -2199,7 +2200,7 @@ int read_loose_object(const char *path,\n \n \t*contents = NULL;\n \n-\tmap = map_sha1_file_1(path, NULL, &mapsize);\n+\tmap = map_sha1_file_1(the_repository, path, NULL, &mapsize);\n \tif (!map) {\n \t\terror_errno(\"unable to mmap %s\", path);\n \t\tgoto out;\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339777","messageId":"20180221015430.96054-19-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 18/27] sha1_file: add repository argument to open_sha1_file","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:21Z","receivedAt":"2018-02-21T01:55:21Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the open_sha1_file caller to be\nmore specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex ed297773e5..68aff90792 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -903,7 +903,9 @@ static int stat_sha1_file_the_repository(const unsigned char *sha1,\n * Like stat_sha1_file(), but actually open the object and return the\n * descriptor. See the caveats on the \"path\" parameter above.\n */\n-static int open_sha1_file(const unsigned char *sha1, const char **path)\n+#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)\n+static int open_sha1_file_the_repository(const unsigned char *sha1,\n+\t\t\t\t\t const char **path)\n {\n \tint fd;\n \tstruct alternate_object_database *alt;\n@@ -946,7 +948,7 @@ static void *map_sha1_file_1(const char *path,\n \tif (path)\n \t\tfd = git_open(path);\n \telse\n-\t\tfd = open_sha1_file(sha1, &path);\n+\t\tfd = open_sha1_file(the_repository, sha1, &path);\n \tmap = NULL;\n \tif (fd >= 0) {\n \t\tstruct stat st;\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339778","messageId":"20180221015430.96054-25-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 24/27] sha1_file: allow open_sha1_file to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:27Z","receivedAt":"2018-02-21T01:55:25Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 +++++------\n 1 file changed, 5 insertions(+), 6 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 6879af8993..598acc5410 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -902,9 +902,8 @@ static int stat_sha1_file(struct repository *r, const unsigned char *sha1,\n * Like stat_sha1_file(), but actually open the object and return the\n * descriptor. See the caveats on the \"path\" parameter above.\n */\n-#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)\n-static int open_sha1_file_the_repository(const unsigned char *sha1,\n-\t\t\t\t\t const char **path)\n+static int open_sha1_file(struct repository *r,\n+\t\t\t const unsigned char *sha1, const char **path)\n {\n \tint fd;\n \tstruct alternate_object_database *alt;\n@@ -912,7 +911,7 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(the_repository, &buf, sha1);\n+\tsha1_file_name(r, &buf, sha1);\n \t*path = buf.buf;\n \n \tfd = git_open(*path);\n@@ -920,8 +919,8 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,\n \t\treturn fd;\n \tmost_interesting_errno = errno;\n \n-\tprepare_alt_odb(the_repository);\n-\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n+\tprepare_alt_odb(r);\n+\tfor (alt = r->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tfd = git_open(*path);\n \t\tif (fd >= 0)\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339779","messageId":"20180221015430.96054-23-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 22/27] sha1_file: allow sha1_file_name to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:25Z","receivedAt":"2018-02-21T01:55:28Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 3 +--\n sha1_file.c | 4 ++--\n 2 files changed, 3 insertions(+), 4 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex a37d74a0c5..146358fe33 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -127,8 +127,7 @@ void raw_object_store_clear(struct raw_object_store *o);\n * Put in `buf` the name of the file in the local object database that\n * would be used to store a loose object with the specified sha1.\n */\n-#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)\n-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);\n+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);\n \n #define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)\n void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d1f77a4c80..8fad523d85 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -323,9 +323,9 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)\n \t}\n }\n \n-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)\n+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1)\n {\n-\tstrbuf_addstr(buf, get_object_directory());\n+\tstrbuf_addstr(buf, r->objects.objectdir);\n \tstrbuf_addch(buf, '/');\n \tfill_sha1_path(buf, sha1);\n }\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339780","messageId":"20180221015430.96054-24-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 23/27] sha1_file: allow stat_sha1_file to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:26Z","receivedAt":"2018-02-21T01:55:29Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 +++++------\n 1 file changed, 5 insertions(+), 6 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 8fad523d85..6879af8993 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -874,23 +874,22 @@ int git_open_cloexec(const char *name, int flags)\n * Note that it may point to static storage and is only valid until another\n * call to sha1_file_name(), etc.\n */\n-#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)\n-static int stat_sha1_file_the_repository(const unsigned char *sha1,\n-\t\t\t\t\t struct stat *st, const char **path)\n+static int stat_sha1_file(struct repository *r, const unsigned char *sha1,\n+\t\t\t struct stat *st, const char **path)\n {\n \tstruct alternate_object_database *alt;\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(the_repository, &buf, sha1);\n+\tsha1_file_name(r, &buf, sha1);\n \t*path = buf.buf;\n \n \tif (!lstat(*path, st))\n \t\treturn 0;\n \n-\tprepare_alt_odb(the_repository);\n+\tprepare_alt_odb(r);\n \terrno = ENOENT;\n-\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = r->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tif (!lstat(*path, st))\n \t\t\treturn 0;\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339781","messageId":"20180221015430.96054-28-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 27/27] sha1_file: allow sha1_loose_object_info to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:30Z","receivedAt":"2018-02-21T01:55:31Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Jonathan Nieder \n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 +++++------\n 1 file changed, 5 insertions(+), 6 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex a20f811a07..36282acb1a 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -1156,10 +1156,9 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)\n \treturn parse_sha1_header_extended(hdr, &oi, 0);\n }\n \n-#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)\n-static int sha1_loose_object_info_the_repository(const unsigned char *sha1,\n-\t\t\t\t\t\t struct object_info *oi,\n-\t\t\t\t\t\t int flags)\n+static int sha1_loose_object_info(struct repository *r,\n+\t\t\t\t const unsigned char *sha1,\n+\t\t\t\t struct object_info *oi, int flags)\n {\n \tint status = 0;\n \tunsigned long mapsize;\n@@ -1183,14 +1182,14 @@ static int sha1_loose_object_info_the_repository(const unsigned char *sha1,\n \tif (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {\n \t\tconst char *path;\n \t\tstruct stat st;\n-\t\tif (stat_sha1_file(the_repository, sha1, &st, &path) < 0)\n+\t\tif (stat_sha1_file(r, sha1, &st, &path) < 0)\n \t\t\treturn -1;\n \t\tif (oi->disk_sizep)\n \t\t\t*oi->disk_sizep = st.st_size;\n \t\treturn 0;\n \t}\n \n-\tmap = map_sha1_file(the_repository, sha1, &mapsize);\n+\tmap = map_sha1_file(r, sha1, &mapsize);\n \tif (!map)\n \t\treturn -1;\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339782","messageId":"20180221015430.96054-27-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 26/27] sha1_file: allow map_sha1_file to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:29Z","receivedAt":"2018-02-21T01:55:33Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 3 +--\n sha1_file.c | 5 +++--\n 2 files changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 146358fe33..5d93677783 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -129,7 +129,6 @@ void raw_object_store_clear(struct raw_object_store *o);\n */\n void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);\n \n-#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)\n-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);\n+void *map_sha1_file(struct repository *r, const unsigned char *sha1, unsigned long *size);\n \n #endif /* OBJECT_STORE_H */\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 522804d9f0..a20f811a07 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -964,9 +964,10 @@ static void *map_sha1_file_1(struct repository *r, const char *path,\n \treturn map;\n }\n \n-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)\n+void *map_sha1_file(struct repository *r,\n+\t\t const unsigned char *sha1, unsigned long *size)\n {\n-\treturn map_sha1_file_1(the_repository, NULL, sha1, size);\n+\treturn map_sha1_file_1(r, NULL, sha1, size);\n }\n \n static int unpack_sha1_short_header(git_zstream *stream,\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339783","messageId":"20180221015430.96054-26-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 25/27] sha1_file: allow map_sha1_file_1 to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:28Z","receivedAt":"2018-02-21T01:55:39Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Jonathan Nieder \n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 8 +++-----\n 1 file changed, 3 insertions(+), 5 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 598acc5410..522804d9f0 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -936,10 +936,8 @@ static int open_sha1_file(struct repository *r,\n * Map the loose object at \"path\" if it is not NULL, or the path found by\n * searching for a loose object named \"sha1\".\n */\n-#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)\n-static void *map_sha1_file_1_the_repository(const char *path,\n-\t\t\t\t\t const unsigned char *sha1,\n-\t\t\t\t\t unsigned long *size)\n+static void *map_sha1_file_1(struct repository *r, const char *path,\n+\t\t\t const unsigned char *sha1, unsigned long *size)\n {\n \tvoid *map;\n \tint fd;\n@@ -947,7 +945,7 @@ static void *map_sha1_file_1_the_repository(const char *path,\n \tif (path)\n \t\tfd = git_open(path);\n \telse\n-\t\tfd = open_sha1_file(the_repository, sha1, &path);\n+\t\tfd = open_sha1_file(r, sha1, &path);\n \tmap = NULL;\n \tif (fd >= 0) {\n \t\tstruct stat st;\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339784","messageId":"20180221015430.96054-21-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 20/27] sha1_file: add repository argument to map_sha1_file","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:23Z","receivedAt":"2018-02-21T01:55:43Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow map_sha1_file callers to be more\nspecific about which repository to handle. This is a small mechanical\nchange; it doesn't change the implementation to handle repositories\nother than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nWhile at it, move the declaration to object-store.h, where it should\nbe easier to find.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n cache.h | 1 -\n object-store.h | 3 +++\n sha1_file.c | 4 ++--\n streaming.c | 5 ++++-\n 4 files changed, 9 insertions(+), 4 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex e641c23b0a..d338b65d75 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1242,7 +1242,6 @@ extern int pretend_sha1_file(void *, unsigned long, enum object_type, unsigned c\n extern int force_object_loose(const unsigned char *sha1, time_t mtime);\n extern int git_open_cloexec(const char *name, int flags);\n #define git_open(name) git_open_cloexec(name, O_RDONLY)\n-extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size);\n extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);\n extern int parse_sha1_header(const char *hdr, unsigned long *sizep);\n \ndiff --git a/object-store.h b/object-store.h\nindex 4f5060e768..a37d74a0c5 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -130,4 +130,7 @@ void raw_object_store_clear(struct raw_object_store *o);\n #define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)\n void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);\n \n+#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)\n+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);\n+\n #endif /* OBJECT_STORE_H */\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 7fa9ed2f25..0d3ff99cf2 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -968,7 +968,7 @@ static void *map_sha1_file_1_the_repository(const char *path,\n \treturn map;\n }\n \n-void *map_sha1_file(const unsigned char *sha1, unsigned long *size)\n+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)\n {\n \treturn map_sha1_file_1(the_repository, NULL, sha1, size);\n }\n@@ -1192,7 +1192,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,\n \t\treturn 0;\n \t}\n \n-\tmap = map_sha1_file(sha1, &mapsize);\n+\tmap = map_sha1_file(the_repository, sha1, &mapsize);\n \tif (!map)\n \t\treturn -1;\n \ndiff --git a/streaming.c b/streaming.c\nindex 5892b50bd8..22d27df55e 100644\n--- a/streaming.c\n+++ b/streaming.c\n@@ -3,6 +3,8 @@\n */\n #include \"cache.h\"\n #include \"streaming.h\"\n+#include \"repository.h\"\n+#include \"object-store.h\"\n #include \"packfile.h\"\n \n enum input_source {\n@@ -335,7 +337,8 @@ static struct stream_vtbl loose_vtbl = {\n \n static open_method_decl(loose)\n {\n-\tst->u.loose.mapped = map_sha1_file(sha1, &st->u.loose.mapsize);\n+\tst->u.loose.mapped = map_sha1_file(the_repository,\n+\t\t\t\t\t sha1, &st->u.loose.mapsize);\n \tif (!st->u.loose.mapped)\n \t\treturn -1;\n \tif ((unpack_sha1_header(&st->z,\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339785","messageId":"20180221015430.96054-22-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 21/27] sha1_file: add repository argument to sha1_loose_object_info","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:24Z","receivedAt":"2018-02-21T01:55:45Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the sha1_loose_object_info caller\nto be more specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 0d3ff99cf2..d1f77a4c80 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -1159,9 +1159,10 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)\n \treturn parse_sha1_header_extended(hdr, &oi, 0);\n }\n \n-static int sha1_loose_object_info(const unsigned char *sha1,\n-\t\t\t\t struct object_info *oi,\n-\t\t\t\t int flags)\n+#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)\n+static int sha1_loose_object_info_the_repository(const unsigned char *sha1,\n+\t\t\t\t\t\t struct object_info *oi,\n+\t\t\t\t\t\t int flags)\n {\n \tint status = 0;\n \tunsigned long mapsize;\n@@ -1280,7 +1281,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,\n \t\t\tbreak;\n \n \t\t/* Most likely it's a loose object. */\n-\t\tif (!sha1_loose_object_info(real, oi, flags))\n+\t\tif (!sha1_loose_object_info(the_repository, real, oi, flags))\n \t\t\treturn 0;\n \n \t\t/* Not a loose object; someone else may have just packed it. */\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339786","messageId":"20180221015430.96054-13-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 12/27] sha1_file: add repository argument to link_alt_odb_entries","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:15Z","receivedAt":"2018-02-21T01:56:02Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"See previous patch for explanation.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 19 +++++++++++++------\n 1 file changed, 13 insertions(+), 6 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 6edcc3d3a7..a0e9116318 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -467,8 +467,12 @@ static const char *parse_alt_odb_entry(const char *string,\n \treturn end;\n }\n \n-static void link_alt_odb_entries(const char *alt, int sep,\n-\t\t\t\t const char *relative_base, int depth)\n+#define link_alt_odb_entries(r, a, s, rb, d) \\\n+\tlink_alt_odb_entries_##r(a, s, rb, d)\n+static void link_alt_odb_entries_the_repository(const char *alt,\n+\t\t\t\t\t\tint sep,\n+\t\t\t\t\t\tconst char *relative_base,\n+\t\t\t\t\t\tint depth)\n {\n \tstruct strbuf objdirbuf = STRBUF_INIT;\n \tstruct strbuf entry = STRBUF_INIT;\n@@ -511,7 +515,7 @@ static void read_info_alternates_the_repository(const char *relative_base,\n \t\treturn;\n \t}\n \n-\tlink_alt_odb_entries(buf.buf, '\\n', relative_base, depth);\n+\tlink_alt_odb_entries(the_repository, buf.buf, '\\n', relative_base, depth);\n \tstrbuf_release(&buf);\n \tfree(path);\n }\n@@ -565,7 +569,8 @@ void add_to_alternates_file(const char *reference)\n \t\tif (commit_lock_file(&lock))\n \t\t\tdie_errno(\"unable to move new alternates file into place\");\n \t\tif (the_repository->objects.alt_odb_tail)\n-\t\t\tlink_alt_odb_entries(reference, '\\n', NULL, 0);\n+\t\t\tlink_alt_odb_entries(the_repository, reference,\n+\t\t\t\t\t '\\n', NULL, 0);\n \t}\n \tfree(alts);\n }\n@@ -578,7 +583,8 @@ void add_to_alternates_memory(const char *reference)\n \t */\n \tprepare_alt_odb();\n \n-\tlink_alt_odb_entries(reference, '\\n', NULL, 0);\n+\tlink_alt_odb_entries(the_repository, reference,\n+\t\t\t '\\n', NULL, 0);\n }\n \n /*\n@@ -681,7 +687,8 @@ void prepare_alt_odb(void)\n \n \tthe_repository->objects.alt_odb_tail =\n \t\t\t&the_repository->objects.alt_odb_list;\n-\tlink_alt_odb_entries(alt, PATH_SEP, NULL, 0);\n+\tlink_alt_odb_entries(the_repository, alt,\n+\t\t\t PATH_SEP, NULL, 0);\n \n \tread_info_alternates(the_repository, get_object_directory(), 0);\n }\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339787","messageId":"20180221015430.96054-18-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 17/27] sha1_file: add repository argument to stat_sha1_file","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:20Z","receivedAt":"2018-02-21T01:56:06Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the stat_sha1_file caller to be\nmore specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 811ed9e53f..ed297773e5 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -874,8 +874,9 @@ int git_open_cloexec(const char *name, int flags)\n * Note that it may point to static storage and is only valid until another\n * call to sha1_file_name(), etc.\n */\n-static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n-\t\t\t const char **path)\n+#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)\n+static int stat_sha1_file_the_repository(const unsigned char *sha1,\n+\t\t\t\t\t struct stat *st, const char **path)\n {\n \tstruct alternate_object_database *alt;\n \tstatic struct strbuf buf = STRBUF_INIT;\n@@ -1181,7 +1182,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,\n \tif (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {\n \t\tconst char *path;\n \t\tstruct stat st;\n-\t\tif (stat_sha1_file(sha1, &st, &path) < 0)\n+\t\tif (stat_sha1_file(the_repository, sha1, &st, &path) < 0)\n \t\t\treturn -1;\n \t\tif (oi->disk_sizep)\n \t\t\t*oi->disk_sizep = st.st_size;\n@@ -1395,7 +1396,7 @@ void *read_sha1_file_extended(const unsigned char *sha1,\n \t\tdie(\"replacement %s not found for %s\",\n \t\t sha1_to_hex(repl), sha1_to_hex(sha1));\n \n-\tif (!stat_sha1_file(repl, &st, &path))\n+\tif (!stat_sha1_file(the_repository, repl, &st, &path))\n \t\tdie(\"loose object %s (stored in %s) is corrupt\",\n \t\t sha1_to_hex(repl), path);\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339788","messageId":"20180221015430.96054-4-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 03/27] object-store: move alt_odb_list and alt_odb_tail to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:06Z","receivedAt":"2018-02-21T01:56:12Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a process with multiple repositories open, alternates should be\nassociated to a single repository and not shared globally. Move\nalt_odb_list and alt_odb_tail into the_repository and adjust callers\nto reflect this.\n\nNow that the alternative object data base is per repository, we're\nleaking its memory upon freeing a repository. The next patch plugs\nthis hole.\n\nNo functional change intended.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/fsck.c | 4 +++-\n object-store.h | 9 ++++++---\n packfile.c | 3 ++-\n sha1_file.c | 25 ++++++++++++-------------\n sha1_name.c | 3 ++-\n 5 files changed, 25 insertions(+), 19 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 7a8a679d4f..908e4f092a 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -1,5 +1,6 @@\n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n #include \"commit.h\"\n #include \"tree.h\"\n@@ -716,7 +717,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t\tfsck_object_dir(get_object_directory());\n \n \t\tprepare_alt_odb();\n-\t\tfor (alt = alt_odb_list; alt; alt = alt->next)\n+\t\tfor (alt = the_repository->objects.alt_odb_list;\n+\t\t\t\talt; alt = alt->next)\n \t\t\tfsck_object_dir(alt->path);\n \n \t\tif (check_full) {\ndiff --git a/object-store.h b/object-store.h\nindex 5678aa1136..e78eea1dde 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -1,7 +1,7 @@\n #ifndef OBJECT_STORE_H\n #define OBJECT_STORE_H\n \n-extern struct alternate_object_database {\n+struct alternate_object_database {\n \tstruct alternate_object_database *next;\n \n \t/* see alt_scratch_buf() */\n@@ -19,7 +19,7 @@ extern struct alternate_object_database {\n \tstruct oid_array loose_objects_cache;\n \n \tchar path[FLEX_ARRAY];\n-} *alt_odb_list;\n+};\n void prepare_alt_odb(void);\n char *compute_alternate_path(const char *path, struct strbuf *err);\n typedef int alt_odb_fn(struct alternate_object_database *, void *);\n@@ -58,8 +58,11 @@ struct raw_object_store {\n \t * Cannot be NULL after initialization.\n \t */\n \tchar *objectdir;\n+\n+\tstruct alternate_object_database *alt_odb_list;\n+\tstruct alternate_object_database **alt_odb_tail;\n };\n-#define RAW_OBJECT_STORE_INIT { NULL }\n+#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/packfile.c b/packfile.c\nindex 7dbe8739d1..216ea836ee 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -1,6 +1,7 @@\n #include \"cache.h\"\n #include \"list.h\"\n #include \"pack.h\"\n+#include \"repository.h\"\n #include \"dir.h\"\n #include \"mergesort.h\"\n #include \"packfile.h\"\n@@ -891,7 +892,7 @@ void prepare_packed_git(void)\n \t\treturn;\n \tprepare_packed_git_one(get_object_directory(), 1);\n \tprepare_alt_odb();\n-\tfor (alt = alt_odb_list; alt; alt = alt->next)\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)\n \t\tprepare_packed_git_one(alt->path, 0);\n \trearrange_packed_git();\n \tprepare_packed_git_mru();\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 831d9e7343..1348dce68f 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -22,6 +22,7 @@\n #include \"pack-revindex.h\"\n #include \"sha1-lookup.h\"\n #include \"bulk-checkin.h\"\n+#include \"repository.h\"\n #include \"streaming.h\"\n #include \"dir.h\"\n #include \"list.h\"\n@@ -342,9 +343,6 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,\n \treturn buf->buf;\n }\n \n-struct alternate_object_database *alt_odb_list;\n-static struct alternate_object_database **alt_odb_tail;\n-\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\n@@ -364,7 +362,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)\n \t * Prevent the common mistake of listing the same\n \t * thing twice, or object directory itself.\n \t */\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\tif (!fspathcmp(path->buf, alt->path))\n \t\t\treturn 0;\n \t}\n@@ -424,8 +422,8 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,\n \tent = alloc_alt_odb(pathbuf.buf);\n \n \t/* add the alternate entry */\n-\t*alt_odb_tail = ent;\n-\talt_odb_tail = &(ent->next);\n+\t*the_repository->objects.alt_odb_tail = ent;\n+\tthe_repository->objects.alt_odb_tail = &(ent->next);\n \tent->next = NULL;\n \n \t/* recursively add alternates */\n@@ -559,7 +557,7 @@ void add_to_alternates_file(const char *reference)\n \t\tfprintf_or_die(out, \"%s\\n\", reference);\n \t\tif (commit_lock_file(&lock))\n \t\t\tdie_errno(\"unable to move new alternates file into place\");\n-\t\tif (alt_odb_tail)\n+\t\tif (the_repository->objects.alt_odb_tail)\n \t\t\tlink_alt_odb_entries(reference, '\\n', NULL, 0);\n \t}\n \tfree(alts);\n@@ -657,7 +655,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \tint r = 0;\n \n \tprepare_alt_odb();\n-\tfor (ent = alt_odb_list; ent; ent = ent->next) {\n+\tfor (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {\n \t\tr = fn(ent, cb);\n \t\tif (r)\n \t\t\tbreak;\n@@ -669,12 +667,13 @@ void prepare_alt_odb(void)\n {\n \tconst char *alt;\n \n-\tif (alt_odb_tail)\n+\tif (the_repository->objects.alt_odb_tail)\n \t\treturn;\n \n \talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n \n-\talt_odb_tail = &alt_odb_list;\n+\tthe_repository->objects.alt_odb_tail =\n+\t\t\t&the_repository->objects.alt_odb_list;\n \tlink_alt_odb_entries(alt, PATH_SEP, NULL, 0);\n \n \tread_info_alternates(get_object_directory(), 0);\n@@ -718,7 +717,7 @@ static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)\n {\n \tstruct alternate_object_database *alt;\n \tprepare_alt_odb();\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\tconst char *path = alt_sha1_path(alt, sha1);\n \t\tif (check_and_freshen_file(path, freshen))\n \t\t\treturn 1;\n@@ -878,7 +877,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n \n \tprepare_alt_odb();\n \terrno = ENOENT;\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tif (!lstat(*path, st))\n \t\t\treturn 0;\n@@ -908,7 +907,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \tmost_interesting_errno = errno;\n \n \tprepare_alt_odb();\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tfd = git_open(*path);\n \t\tif (fd >= 0)\ndiff --git a/sha1_name.c b/sha1_name.c\nindex 611c7d24dd..957ce25680 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -10,6 +10,7 @@\n #include \"dir.h\"\n #include \"sha1-array.h\"\n #include \"packfile.h\"\n+#include \"repository.h\"\n \n static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);\n \n@@ -104,7 +105,7 @@ static void find_short_object_filename(struct disambiguate_state *ds)\n \t\t */\n \t\tfakeent = alloc_alt_odb(get_object_directory());\n \t}\n-\tfakeent->next = alt_odb_list;\n+\tfakeent->next = the_repository->objects.alt_odb_list;\n \n \tfor (alt = fakeent; alt && !ds->ambiguous; alt = alt->next) {\n \t\tint pos;\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339789","messageId":"20180221015430.96054-17-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 16/27] sha1_file: add repository argument to sha1_file_name","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:19Z","receivedAt":"2018-02-21T01:56:16Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow sha1_file_name callers to be more\nspecific about which repository to handle. This is a small mechanical\nchange; it doesn't change the implementation to handle repositories\nother than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nWhile at it, move the declaration to object-store.h, where it should\nbe easier to find.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n cache.h | 6 ------\n http-walker.c | 3 ++-\n http.c | 5 ++---\n object-store.h | 7 +++++++\n sha1_file.c | 10 +++++-----\n 5 files changed, 16 insertions(+), 15 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 3fd9dc52cd..e641c23b0a 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -961,12 +961,6 @@ extern void check_repository_format(void);\n #define DATA_CHANGED 0x0020\n #define TYPE_CHANGED 0x0040\n \n-/*\n- * Put in `buf` the name of the file in the local object database that\n- * would be used to store a loose object with the specified sha1.\n- */\n-extern void sha1_file_name(struct strbuf *buf, const unsigned char *sha1);\n-\n /*\n * Return an abbreviated sha1 unique within this repository's object database.\n * The result will be at least `len` characters long, and will be NUL\ndiff --git a/http-walker.c b/http-walker.c\nindex 8bb5d991bb..76bfdbca76 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -1,4 +1,5 @@\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"commit.h\"\n #include \"walker.h\"\n #include \"http.h\"\n@@ -546,7 +547,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1)\n \t\tret = error(\"File %s has bad hash\", hex);\n \t} else if (req->rename < 0) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n-\t\tsha1_file_name(&buf, req->sha1);\n+\t\tsha1_file_name(the_repository, &buf, req->sha1);\n \t\tret = error(\"unable to write sha1 filename %s\", buf.buf);\n \t\tstrbuf_release(&buf);\n \t}\ndiff --git a/http.c b/http.c\nindex a4a9e583c7..afe2707e86 100644\n--- a/http.c\n+++ b/http.c\n@@ -2247,7 +2247,7 @@ struct http_object_request *new_http_object_request(const char *base_url,\n \thashcpy(freq->sha1, sha1);\n \tfreq->localfile = -1;\n \n-\tsha1_file_name(&filename, sha1);\n+\tsha1_file_name(the_repository, &filename, sha1);\n \tsnprintf(freq->tmpfile, sizeof(freq->tmpfile),\n \t\t \"%s.temp\", filename.buf);\n \n@@ -2396,8 +2396,7 @@ int finish_http_object_request(struct http_object_request *freq)\n \t\tunlink_or_warn(freq->tmpfile);\n \t\treturn -1;\n \t}\n-\n-\tsha1_file_name(&filename, freq->sha1);\n+\tsha1_file_name(the_repository, &filename, freq->sha1);\n \tfreq->rename = finalize_object_file(freq->tmpfile, filename.buf);\n \tstrbuf_release(&filename);\n \ndiff --git a/object-store.h b/object-store.h\nindex 873b341774..4f5060e768 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -123,4 +123,11 @@ struct raw_object_store {\n \n void raw_object_store_clear(struct raw_object_store *o);\n \n+/*\n+ * Put in `buf` the name of the file in the local object database that\n+ * would be used to store a loose object with the specified sha1.\n+ */\n+#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)\n+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);\n+\n #endif /* OBJECT_STORE_H */\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 9cf3fffbeb..811ed9e53f 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -323,7 +323,7 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)\n \t}\n }\n \n-void sha1_file_name(struct strbuf *buf, const unsigned char *sha1)\n+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)\n {\n \tstrbuf_addstr(buf, get_object_directory());\n \tstrbuf_addch(buf, '/');\n@@ -720,7 +720,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(&buf, sha1);\n+\tsha1_file_name(the_repository, &buf, sha1);\n \n \treturn check_and_freshen_file(buf.buf, freshen);\n }\n@@ -881,7 +881,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(&buf, sha1);\n+\tsha1_file_name(the_repository, &buf, sha1);\n \t*path = buf.buf;\n \n \tif (!lstat(*path, st))\n@@ -910,7 +910,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(&buf, sha1);\n+\tsha1_file_name(the_repository, &buf, sha1);\n \t*path = buf.buf;\n \n \tfd = git_open(*path);\n@@ -1595,7 +1595,7 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,\n \tstatic struct strbuf filename = STRBUF_INIT;\n \n \tstrbuf_reset(&filename);\n-\tsha1_file_name(&filename, sha1);\n+\tsha1_file_name(the_repository, &filename, sha1);\n \n \tfd = create_tmpfile(&tmp_file, filename.buf);\n \tif (fd < 0) {\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339790","messageId":"20180221015430.96054-5-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 04/27] object-store: free alt_odb_list","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:07Z","receivedAt":"2018-02-21T01:56:19Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Free the memory and reset alt_odb_{list, tail} to NULL.\n\nSigned-off-by: Stefan Beller \n---\n object.c | 17 +++++++++++++++++\n 1 file changed, 17 insertions(+)\n\ndiff --git a/object.c b/object.c\nindex 11d904c033..17b1da6d6b 100644\n--- a/object.c\n+++ b/object.c\n@@ -446,7 +446,24 @@ void clear_commit_marks_all(unsigned int flags)\n \t}\n }\n \n+static void free_alt_odb(struct alternate_object_database *alt)\n+{\n+\tstrbuf_release(&alt->scratch);\n+\toid_array_clear(&alt->loose_objects_cache);\n+}\n+\n+static void free_alt_odbs(struct raw_object_store *o)\n+{\n+\twhile (o->alt_odb_list) {\n+\t\tfree_alt_odb(o->alt_odb_list);\n+\t\to->alt_odb_list = o->alt_odb_list->next;\n+\t}\n+}\n+\n void raw_object_store_clear(struct raw_object_store *o)\n {\n \tfree(o->objectdir);\n+\n+\tfree_alt_odbs(o);\n+\to->alt_odb_tail = NULL;\n }\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339791","messageId":"20180221015430.96054-8-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 07/27] pack: move prepare_packed_git_run_once to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:10Z","receivedAt":"2018-02-21T01:56:20Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Each repository's object store can be initialized independently, so\nthey must not share a run_once variable.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 8 +++++++-\n packfile.c | 7 +++----\n 2 files changed, 10 insertions(+), 5 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 1de9e07102..6cecba3951 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -92,6 +92,12 @@ struct raw_object_store {\n \n \tstruct alternate_object_database *alt_odb_list;\n \tstruct alternate_object_database **alt_odb_tail;\n+\n+\t/*\n+\t * Whether packed_git has already been populated with this repository's\n+\t * packs.\n+\t */\n+\tunsigned packed_git_initialized : 1;\n };\n \n /*\n@@ -101,7 +107,7 @@ struct raw_object_store {\n * that macro on member variables. Use NULL instead as that is defined\n * and accepted, deferring the real init to prepare_packed_git_mru(). */\n #define __MRU_INIT { NULL, NULL }\n-#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL }\n+#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0 }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/packfile.c b/packfile.c\nindex 511a2b0cdf..a8a2e7fe43 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -884,12 +884,11 @@ static void prepare_packed_git_mru(void)\n \t\tlist_add_tail(&p->mru, &the_repository->objects.packed_git_mru);\n }\n \n-static int prepare_packed_git_run_once = 0;\n void prepare_packed_git(void)\n {\n \tstruct alternate_object_database *alt;\n \n-\tif (prepare_packed_git_run_once)\n+\tif (the_repository->objects.packed_git_initialized)\n \t\treturn;\n \tprepare_packed_git_one(get_object_directory(), 1);\n \tprepare_alt_odb();\n@@ -897,13 +896,13 @@ void prepare_packed_git(void)\n \t\tprepare_packed_git_one(alt->path, 0);\n \trearrange_packed_git();\n \tprepare_packed_git_mru();\n-\tprepare_packed_git_run_once = 1;\n+\tthe_repository->objects.packed_git_initialized = 1;\n }\n \n void reprepare_packed_git(void)\n {\n \tapproximate_object_count_valid = 0;\n-\tprepare_packed_git_run_once = 0;\n+\tthe_repository->objects.packed_git_initialized = 0;\n \tprepare_packed_git();\n }\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339792","messageId":"20180221015430.96054-7-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCH 06/27] object-store: close all packs upon clearing the object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-21T01:54:09Z","receivedAt":"2018-02-21T01:56:23Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n builtin/am.c | 2 +-\n builtin/clone.c | 2 +-\n builtin/fetch.c | 2 +-\n builtin/merge.c | 2 +-\n builtin/receive-pack.c | 2 +-\n object.c | 6 ++----\n packfile.c | 4 ++--\n packfile.h | 2 +-\n 8 files changed, 10 insertions(+), 12 deletions(-)\n\ndiff --git a/builtin/am.c b/builtin/am.c\nindex 5bdd2d7578..4762a702e3 100644\n--- a/builtin/am.c\n+++ b/builtin/am.c\n@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)\n \t */\n \tif (!state->rebasing) {\n \t\tam_destroy(state);\n-\t\tclose_all_packs();\n+\t\tclose_all_packs(&the_repository->objects);\n \t\trun_command_v_opt(argv_gc_auto, RUN_GIT_CMD);\n \t}\n }\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 101c27a593..13cfaa6488 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -1217,7 +1217,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \ttransport_disconnect(transport);\n \n \tif (option_dissociate) {\n-\t\tclose_all_packs();\n+\t\tclose_all_packs(&the_repository->objects);\n \t\tdissociate_from_references();\n \t}\n \ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 8ee998ea2e..4d72efca78 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -1478,7 +1478,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n \n \tstring_list_clear(&list, 0);\n \n-\tclose_all_packs();\n+\tclose_all_packs(&the_repository->objects);\n \n \targv_array_pushl(&argv_gc_auto, \"gc\", \"--auto\", NULL);\n \tif (verbosity < 0)\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 30264cfd7c..907ae44ab5 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,\n \t\t\t * We ignore errors in 'gc --auto', since the\n \t\t\t * user should see them.\n \t\t\t */\n-\t\t\tclose_all_packs();\n+\t\t\tclose_all_packs(&the_repository->objects);\n \t\t\trun_command_v_opt(argv_gc_auto, RUN_GIT_CMD);\n \t\t}\n \t}\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex b2eac79a6e..954fc72c7c 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -2027,7 +2027,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)\n \t\t\tproc.git_cmd = 1;\n \t\t\tproc.argv = argv_gc_auto;\n \n-\t\t\tclose_all_packs();\n+\t\t\tclose_all_packs(&the_repository->objects);\n \t\t\tif (!start_command(&proc)) {\n \t\t\t\tif (use_sideband)\n \t\t\t\t\tcopy_to_sideband(proc.err, -1, NULL);\ndiff --git a/object.c b/object.c\nindex c76b62572a..34daaf37b3 100644\n--- a/object.c\n+++ b/object.c\n@@ -4,6 +4,7 @@\n #include \"tree.h\"\n #include \"commit.h\"\n #include \"tag.h\"\n+#include \"packfile.h\"\n \n static struct object **obj_hash;\n static int nr_objs, obj_hash_size;\n@@ -469,8 +470,5 @@ void raw_object_store_clear(struct raw_object_store *o)\n \n \twhile (!list_empty(&o->packed_git_mru))\n \t\tlist_del(&o->packed_git_mru);\n-\t/*\n-\t * TODO: call close_all_packs once migrated to\n-\t * take an object store argument\n-\t */\n+\tclose_all_packs(o);\n }\ndiff --git a/packfile.c b/packfile.c\nindex d41e4c83d0..511a2b0cdf 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -311,11 +311,11 @@ static void close_pack(struct packed_git *p)\n \tclose_pack_index(p);\n }\n \n-void close_all_packs(void)\n+void close_all_packs(struct raw_object_store *o)\n {\n \tstruct packed_git *p;\n \n-\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n+\tfor (p = o->packed_git; p; p = p->next)\n \t\tif (p->do_not_close)\n \t\t\tdie(\"BUG: want to close pack marked 'do-not-close'\");\n \t\telse\ndiff --git a/packfile.h b/packfile.h\nindex a7fca598d6..6a2c57045c 100644\n--- a/packfile.h\n+++ b/packfile.h\n@@ -63,7 +63,7 @@ extern void close_pack_index(struct packed_git *);\n \n extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);\n extern void close_pack_windows(struct packed_git *);\n-extern void close_all_packs(void);\n+extern void close_all_packs(struct raw_object_store *o);\n extern void unuse_pack(struct pack_window **);\n extern void clear_delta_base_cache(void);\n extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"339842","messageId":"xmqqy3jmxb8s.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"20180221015430.96054-6-sbeller@google.com","subject":"Re: [PATCH 05/27] object-store: move packed_git and packed_git_mru to object store","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-21T21:51:15Z","receivedAt":"2018-02-21T21:51:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> +\n> +/*\n> + * The mru list_head is supposed to be initialized using\n> + * the LIST_HEAD macro, assigning prev/next to itself.\n> + * However this doesn't work in this case as some compilers dislike\n> + * that macro on member variables. Use NULL instead as that is defined\n> + * and accepted, deferring the real init to prepare_packed_git_mru(). */\n> +#define __MRU_INIT { NULL, NULL }\n> +#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL }\n\nI do not think it has to be this way to abuse two NULLs, if you\nfaithfully mimicked how LIST_HEAD() macro is constructed. The\nreason why it does not try to introduce\n\n\tstruct list_head x = LIST_HEAD_INIT;\n\nand instead, uses\n\n\tLIST_HEAD(x);\n\nis because of the need for self referral. If we learn from it, we\ncan do the same, i.e. instead of doing\n\n\tstruct raw_object_store x = RAW_OBJECT_STORE_INIT;\n\nwe can do\n\n\tRAW_OBJECT_STORE(x);\n\nthat expands to\n\n\tstruct raw_object_store x = {\n\t\t...\n\t\t{ &x.packed_git_mru, &x.packed_git_mru },\n\t\t...\n\t};\n\nno? Then we do not need such a lengthy comment that reads only like\nan excuse ;-)\n\n"},{"id":"339876","messageId":"20180221162019.bba08985870de3755a550b57@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-8-sbeller@google.com","subject":"Re: [PATCH 07/27] pack: move prepare_packed_git_run_once to object store","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-02-22T00:20:19Z","receivedAt":"2018-02-22T00:20:26Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Tue, 20 Feb 2018 17:54:10 -0800\nStefan Beller wrote:\n\n> Each repository's object store can be initialized independently, so\n> they must not share a run_once variable.\n> \n> Signed-off-by: Stefan Beller \n> Signed-off-by: Jonathan Nieder \n\nReviewed-by: Jonathan Tan \n\n> -#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL }\n> +#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0 }\n\nOptional: Trailing zeros are not needed in struct initializers.\n"},{"id":"339880","messageId":"CAGZ79kbZzVDyNxL6fBAfdL4hjjtq=eDjVu3A_zohZJk-bKW9uw@mail.gmail.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"Re: [PATCHv3 00/27] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-22T00:26:19Z","receivedAt":"2018-02-22T00:26:25Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Feb 20, 2018 at 5:54 PM, Stefan Beller wrote:\n> v3:\n> * reverted back to use the repository for most of the functions\n> (including unduplicating 'ignore_env')\n> * rebased on master again (I lost that state when doing v2, as\n> I did both rebase as well as conversion to object store in one go)\n> * one additional patch, that moves the alternates related things to\n> object-store.h, such that inclusion of cache.h (in object-store.h)\n> is not required any more.\n\nThis is also available at\nhttps://github.com/stefanbeller/git/tree/object-store-part1-v3\n"},{"id":"339881","messageId":"20180221162904.54c0d57808d1042c578b8778@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-10-sbeller@google.com","subject":"Re: [PATCH 09/27] sha1_file: add raw_object_store argument to alt_odb_usable","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-02-22T00:29:04Z","receivedAt":"2018-02-22T00:29:12Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Tue, 20 Feb 2018 17:54:12 -0800\nStefan Beller wrote:\n\n> Add a raw_object_store to alt_odb_usable to be more specific about which\n> repository to act on. The choice of the repository is delegated to its\n> only caller link_alt_odb_entry.\n> \n> Signed-off-by: Stefan Beller \n\nI checked that alt_odb_usable no longer depends on any repository-like\nglobals.\n\nReviewed-by: Jonathan Tan \n"},{"id":"339883","messageId":"20180221163530.3a6659b8e0bec5dfab835232@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-16-sbeller@google.com","subject":"Re: [PATCH 15/27] sha1_file: allow prepare_alt_odb to handle arbitrary repositories","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-02-22T00:35:30Z","receivedAt":"2018-02-22T00:35:38Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Tue, 20 Feb 2018 17:54:18 -0800\nStefan Beller wrote:\n\n> -void prepare_alt_odb_the_repository(void)\n> +void prepare_alt_odb(struct repository *r)\n> {\n> -\tconst char *alt;\n> -\n> -\tif (the_repository->objects.alt_odb_tail)\n> +\tif (r->objects.alt_odb_tail)\n> \t\treturn;\n> \n> -\talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n> +\tr->objects.alt_odb_tail = &r->objects.alt_odb_list;\n> +\n> +\tif (!r->ignore_env) {\n> +\t\tconst char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);\n> +\t\tif (!alt)\n> +\t\t\talt = \"\";\n\nalt can be NULL, just like in the existing code, so these 2 lines are\nunnecessary. (link_alt_odb_entries() will just do nothing in either\ncase.)\n\n(I also think that the check of absence of alt should be done by the\ncaller of link_alt_odb_entries(), not by link_alt_odb_entries() itself,\nbut that is much beyond the scope of this patch set.)\n\n> \n> -\tthe_repository->objects.alt_odb_tail =\n> -\t\t\t&the_repository->objects.alt_odb_list;\n> -\tlink_alt_odb_entries(the_repository, alt,\n> -\t\t\t PATH_SEP, NULL, 0);\n> +\t\tlink_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);\n> +\t}\n> \n> -\tread_info_alternates(the_repository, get_object_directory(), 0);\n> +\tread_info_alternates(r, r->objects.objectdir, 0);\n> }\n"},{"id":"339884","messageId":"20180221163630.a69b33acadf774753cf747ba@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-20-sbeller@google.com","subject":"Re: [PATCH 19/27] sha1_file: add repository argument to map_sha1_file_1","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-02-22T00:36:30Z","receivedAt":"2018-02-22T00:36:37Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Tue, 20 Feb 2018 17:54:22 -0800\nStefan Beller wrote:\n\n> Add a repository argument to allow the map_sha1_file_1 caller to be\n> more specific about which repository to act on. This is a small\n> mechanical change; it doesn't change the implementation to handle\n> repositories other than the_repository yet.\n> \n> As with the previous commits, use a macro to catch callers passing a\n> repository other than the_repository at compile time.\n> \n> Signed-off-by: Stefan Beller \n> Signed-off-by: Jonathan Nieder \n\nReviewed-by: Jonathan Tan \n"},{"id":"339885","messageId":"20180222003758.GB127348@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-3-sbeller@google.com","subject":"Re: [PATCH 02/27] object-store: migrate alternates struct and functions from cache.h","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-22T00:37:58Z","receivedAt":"2018-02-22T00:38:06Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/20, Stefan Beller wrote:\n> Migrate the struct alternate_object_database and all its related\n> functions to the object store as these functions are easier found in\n> that header. The migration is just a verbatim copy, no need to\n> include the object store header at any C file, because cache.h includes\n> repository.h which in turn includes the object-store.h\n\nNice. Always love seeing cache.h get smaller\n\n\n-- \nBrandon Williams\n"},{"id":"339886","messageId":"20180222004127.GC127348@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-4-sbeller@google.com","subject":"Re: [PATCH 03/27] object-store: move alt_odb_list and alt_odb_tail to object store","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-22T00:41:27Z","receivedAt":"2018-02-22T00:41:34Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/20, Stefan Beller wrote:\n> In a process with multiple repositories open, alternates should be\n> associated to a single repository and not shared globally. Move\n> alt_odb_list and alt_odb_tail into the_repository and adjust callers\n> to reflect this.\n> \n> Now that the alternative object data base is per repository, we're\n> leaking its memory upon freeing a repository. The next patch plugs\n> this hole.\n> \n> No functional change intended.\n> \n> Signed-off-by: Stefan Beller \n> Signed-off-by: Jonathan Nieder \n\nLooks good aside from the one spacing issue (though it's not that big of\na deal).\n\n> ---\n> builtin/fsck.c | 4 +++-\n> object-store.h | 9 ++++++---\n> packfile.c | 3 ++-\n> sha1_file.c | 25 ++++++++++++-------------\n> sha1_name.c | 3 ++-\n> 5 files changed, 25 insertions(+), 19 deletions(-)\n> \n> diff --git a/builtin/fsck.c b/builtin/fsck.c\n> index 7a8a679d4f..908e4f092a 100644\n> --- a/builtin/fsck.c\n> +++ b/builtin/fsck.c\n> @@ -1,5 +1,6 @@\n> #include \"builtin.h\"\n> #include \"cache.h\"\n> +#include \"repository.h\"\n> #include \"config.h\"\n> #include \"commit.h\"\n> #include \"tree.h\"\n> @@ -716,7 +717,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n> \t\tfsck_object_dir(get_object_directory());\n> \n> \t\tprepare_alt_odb();\n> -\t\tfor (alt = alt_odb_list; alt; alt = alt->next)\n> +\t\tfor (alt = the_repository->objects.alt_odb_list;\n> +\t\t\t\talt; alt = alt->next)\n\nIndentation on this line looks odd.\n\n\n-- \nBrandon Williams\n"},{"id":"339887","messageId":"20180222004244.GD127348@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-5-sbeller@google.com","subject":"Re: [PATCH 04/27] object-store: free alt_odb_list","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-22T00:42:44Z","receivedAt":"2018-02-22T00:42:52Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/20, Stefan Beller wrote:\n> Free the memory and reset alt_odb_{list, tail} to NULL.\n\nGood to see memory leaks being avoided (well they will be on other\nrepository objects)\n\n> \n> Signed-off-by: Stefan Beller \n> ---\n> object.c | 17 +++++++++++++++++\n> 1 file changed, 17 insertions(+)\n> \n> diff --git a/object.c b/object.c\n> index 11d904c033..17b1da6d6b 100644\n> --- a/object.c\n> +++ b/object.c\n> @@ -446,7 +446,24 @@ void clear_commit_marks_all(unsigned int flags)\n> \t}\n> }\n> \n> +static void free_alt_odb(struct alternate_object_database *alt)\n> +{\n> +\tstrbuf_release(&alt->scratch);\n> +\toid_array_clear(&alt->loose_objects_cache);\n> +}\n> +\n> +static void free_alt_odbs(struct raw_object_store *o)\n> +{\n> +\twhile (o->alt_odb_list) {\n> +\t\tfree_alt_odb(o->alt_odb_list);\n> +\t\to->alt_odb_list = o->alt_odb_list->next;\n> +\t}\n> +}\n> +\n> void raw_object_store_clear(struct raw_object_store *o)\n> {\n> \tfree(o->objectdir);\n> +\n> +\tfree_alt_odbs(o);\n> +\to->alt_odb_tail = NULL;\n> }\n> -- \n> 2.16.1.291.g4437f3f132-goog\n> \n\n-- \nBrandon Williams\n"},{"id":"339888","messageId":"20180222004352.GE127348@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-7-sbeller@google.com","subject":"Re: [PATCH 06/27] object-store: close all packs upon clearing the object store","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-22T00:43:52Z","receivedAt":"2018-02-22T00:44:01Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/20, Stefan Beller wrote:\n> Signed-off-by: Stefan Beller \n\nStraight forward change, looks good.\n\n> ---\n> builtin/am.c | 2 +-\n> builtin/clone.c | 2 +-\n> builtin/fetch.c | 2 +-\n> builtin/merge.c | 2 +-\n> builtin/receive-pack.c | 2 +-\n> object.c | 6 ++----\n> packfile.c | 4 ++--\n> packfile.h | 2 +-\n> 8 files changed, 10 insertions(+), 12 deletions(-)\n> \n> diff --git a/builtin/am.c b/builtin/am.c\n> index 5bdd2d7578..4762a702e3 100644\n> --- a/builtin/am.c\n> +++ b/builtin/am.c\n> @@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)\n> \t */\n> \tif (!state->rebasing) {\n> \t\tam_destroy(state);\n> -\t\tclose_all_packs();\n> +\t\tclose_all_packs(&the_repository->objects);\n> \t\trun_command_v_opt(argv_gc_auto, RUN_GIT_CMD);\n> \t}\n> }\n> diff --git a/builtin/clone.c b/builtin/clone.c\n> index 101c27a593..13cfaa6488 100644\n> --- a/builtin/clone.c\n> +++ b/builtin/clone.c\n> @@ -1217,7 +1217,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n> \ttransport_disconnect(transport);\n> \n> \tif (option_dissociate) {\n> -\t\tclose_all_packs();\n> +\t\tclose_all_packs(&the_repository->objects);\n> \t\tdissociate_from_references();\n> \t}\n> \n> diff --git a/builtin/fetch.c b/builtin/fetch.c\n> index 8ee998ea2e..4d72efca78 100644\n> --- a/builtin/fetch.c\n> +++ b/builtin/fetch.c\n> @@ -1478,7 +1478,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n> \n> \tstring_list_clear(&list, 0);\n> \n> -\tclose_all_packs();\n> +\tclose_all_packs(&the_repository->objects);\n> \n> \targv_array_pushl(&argv_gc_auto, \"gc\", \"--auto\", NULL);\n> \tif (verbosity < 0)\n> diff --git a/builtin/merge.c b/builtin/merge.c\n> index 30264cfd7c..907ae44ab5 100644\n> --- a/builtin/merge.c\n> +++ b/builtin/merge.c\n> @@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,\n> \t\t\t * We ignore errors in 'gc --auto', since the\n> \t\t\t * user should see them.\n> \t\t\t */\n> -\t\t\tclose_all_packs();\n> +\t\t\tclose_all_packs(&the_repository->objects);\n> \t\t\trun_command_v_opt(argv_gc_auto, RUN_GIT_CMD);\n> \t\t}\n> \t}\n> diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\n> index b2eac79a6e..954fc72c7c 100644\n> --- a/builtin/receive-pack.c\n> +++ b/builtin/receive-pack.c\n> @@ -2027,7 +2027,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)\n> \t\t\tproc.git_cmd = 1;\n> \t\t\tproc.argv = argv_gc_auto;\n> \n> -\t\t\tclose_all_packs();\n> +\t\t\tclose_all_packs(&the_repository->objects);\n> \t\t\tif (!start_command(&proc)) {\n> \t\t\t\tif (use_sideband)\n> \t\t\t\t\tcopy_to_sideband(proc.err, -1, NULL);\n> diff --git a/object.c b/object.c\n> index c76b62572a..34daaf37b3 100644\n> --- a/object.c\n> +++ b/object.c\n> @@ -4,6 +4,7 @@\n> #include \"tree.h\"\n> #include \"commit.h\"\n> #include \"tag.h\"\n> +#include \"packfile.h\"\n> \n> static struct object **obj_hash;\n> static int nr_objs, obj_hash_size;\n> @@ -469,8 +470,5 @@ void raw_object_store_clear(struct raw_object_store *o)\n> \n> \twhile (!list_empty(&o->packed_git_mru))\n> \t\tlist_del(&o->packed_git_mru);\n> -\t/*\n> -\t * TODO: call close_all_packs once migrated to\n> -\t * take an object store argument\n> -\t */\n> +\tclose_all_packs(o);\n> }\n> diff --git a/packfile.c b/packfile.c\n> index d41e4c83d0..511a2b0cdf 100644\n> --- a/packfile.c\n> +++ b/packfile.c\n> @@ -311,11 +311,11 @@ static void close_pack(struct packed_git *p)\n> \tclose_pack_index(p);\n> }\n> \n> -void close_all_packs(void)\n> +void close_all_packs(struct raw_object_store *o)\n> {\n> \tstruct packed_git *p;\n> \n> -\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n> +\tfor (p = o->packed_git; p; p = p->next)\n> \t\tif (p->do_not_close)\n> \t\t\tdie(\"BUG: want to close pack marked 'do-not-close'\");\n> \t\telse\n> diff --git a/packfile.h b/packfile.h\n> index a7fca598d6..6a2c57045c 100644\n> --- a/packfile.h\n> +++ b/packfile.h\n> @@ -63,7 +63,7 @@ extern void close_pack_index(struct packed_git *);\n> \n> extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);\n> extern void close_pack_windows(struct packed_git *);\n> -extern void close_all_packs(void);\n> +extern void close_all_packs(struct raw_object_store *o);\n> extern void unuse_pack(struct pack_window **);\n> extern void clear_delta_base_cache(void);\n> extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);\n> -- \n> 2.16.1.291.g4437f3f132-goog\n> \n\n-- \nBrandon Williams\n"},{"id":"339889","messageId":"20180221164406.c98170765e454b51de80c680@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-23-sbeller@google.com","subject":"Re: [PATCH 22/27] sha1_file: allow sha1_file_name to handle arbitrary repositories","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-02-22T00:44:06Z","receivedAt":"2018-02-22T00:44:18Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Tue, 20 Feb 2018 17:54:25 -0800\nStefan Beller wrote:\n\n> Signed-off-by: Stefan Beller \n> Signed-off-by: Jonathan Nieder \n\nReviewed-by: Jonathan Tan \n\n> -void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)\n> +void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1)\n> {\n> -\tstrbuf_addstr(buf, get_object_directory());\n> +\tstrbuf_addstr(buf, r->objects.objectdir);\n> \tstrbuf_addch(buf, '/');\n> \tfill_sha1_path(buf, sha1);\n> }\n\nIn the future, we should probably have:\n - a function to get the object store out of a repo (so that it can\n lazily initialize the object store struct if necessary)\n - when the object store is obtained, its objectdir field is guaranteed\n to be populated\n - sha1_file_name should take the object store struct, not the repo\n struct\n\nbut this is outside the scope of this patch.\n"},{"id":"339890","messageId":"20180221164508.ebb49caaf24499caf386dae7@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-25-sbeller@google.com","subject":"Re: [PATCH 24/27] sha1_file: allow open_sha1_file to handle arbitrary repositories","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-02-22T00:45:08Z","receivedAt":"2018-02-22T00:45:15Z","isPatch":true,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Tue, 20 Feb 2018 17:54:27 -0800\nStefan Beller wrote:\n\n> Signed-off-by: Stefan Beller \n> Signed-off-by: Jonathan Nieder \n\nReviewed-by: Jonathan Tan \n"},{"id":"339891","messageId":"20180222004710.GF127348@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-9-sbeller@google.com","subject":"Re: [PATCH 08/27] pack: move approximate object count to object store","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-22T00:47:10Z","receivedAt":"2018-02-22T00:47:17Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/20, Stefan Beller wrote:\n> The approximate_object_count() function maintains a rough count of\n> objects in a repository to estimate how long object name abbreviates\n> should be. Object names are scoped to a repository and the\n> appropriate length may differ by repository, so the object count\n> should not be global.\n> \n> Signed-off-by: Stefan Beller \n> Signed-off-by: Jonathan Nieder \n> ---\n> object-store.h | 10 +++++++++-\n> packfile.c | 11 +++++------\n> 2 files changed, 14 insertions(+), 7 deletions(-)\n> \n> diff --git a/object-store.h b/object-store.h\n> index 6cecba3951..bd1e4fcd8b 100644\n> --- a/object-store.h\n> +++ b/object-store.h\n> @@ -93,6 +93,14 @@ struct raw_object_store {\n> \tstruct alternate_object_database *alt_odb_list;\n> \tstruct alternate_object_database **alt_odb_tail;\n> \n> +\t/*\n> +\t * A fast, rough count of the number of objects in the repository.\n> +\t * These two fields are not meant for direct access. Use\n> +\t * approximate_object_count() instead.\n> +\t */\n> +\tunsigned long approximate_object_count;\n> +\tunsigned approximate_object_count_valid : 1;\n\nPatch looks fine and is effectively a no-op, though what is the need for\nboth of these variables? Maybe it can be simplified down to just use\none? Just musing as its out of the scope of this patch and we probably\nshouldn't try to change that in this series.\n\n> +\n> \t/*\n> \t * Whether packed_git has already been populated with this repository's\n> \t * packs.\n> @@ -107,7 +115,7 @@ struct raw_object_store {\n> * that macro on member variables. Use NULL instead as that is defined\n> * and accepted, deferring the real init to prepare_packed_git_mru(). */\n> #define __MRU_INIT { NULL, NULL }\n> -#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0 }\n> +#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL, 0, 0, 0 }\n> \n> void raw_object_store_clear(struct raw_object_store *o);\n> \n> diff --git a/packfile.c b/packfile.c\n> index a8a2e7fe43..693bafbc98 100644\n> --- a/packfile.c\n> +++ b/packfile.c\n> @@ -803,8 +803,6 @@ static void prepare_packed_git_one(char *objdir, int local)\n> \tstrbuf_release(&path);\n> }\n> \n> -static int approximate_object_count_valid;\n> -\n> /*\n> * Give a fast, rough count of the number of objects in the repository. This\n> * ignores loose objects completely. If you have a lot of them, then either\n> @@ -814,8 +812,8 @@ static int approximate_object_count_valid;\n> */\n> unsigned long approximate_object_count(void)\n> {\n> -\tstatic unsigned long count;\n> -\tif (!approximate_object_count_valid) {\n> +\tif (!the_repository->objects.approximate_object_count_valid) {\n> +\t\tunsigned long count;\n> \t\tstruct packed_git *p;\n> \n> \t\tprepare_packed_git();\n> @@ -825,8 +823,9 @@ unsigned long approximate_object_count(void)\n> \t\t\t\tcontinue;\n> \t\t\tcount += p->num_objects;\n> \t\t}\n> +\t\tthe_repository->objects.approximate_object_count = count;\n> \t}\n> -\treturn count;\n> +\treturn the_repository->objects.approximate_object_count;\n> }\n> \n> static void *get_next_packed_git(const void *p)\n> @@ -901,7 +900,7 @@ void prepare_packed_git(void)\n> \n> void reprepare_packed_git(void)\n> {\n> -\tapproximate_object_count_valid = 0;\n> +\tthe_repository->objects.approximate_object_count_valid = 0;\n> \tthe_repository->objects.packed_git_initialized = 0;\n> \tprepare_packed_git();\n> }\n> -- \n> 2.16.1.291.g4437f3f132-goog\n> \n\n-- \nBrandon Williams\n"},{"id":"339892","messageId":"20180222004945.GG127348@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-15-sbeller@google.com","subject":"Re: [PATCH 14/27] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-22T00:49:45Z","receivedAt":"2018-02-22T00:49:53Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/20, Stefan Beller wrote:\n> Actually this also allows read_info_alternates and link_alt_odb_entry to\n> handle arbitrary repositories, but link_alt_odb_entries is the most\n> interesting function in this set of functions, hence the commit subject.\n> \n> These functions span a strongly connected component in the function\n> graph, i.e. the recursive call chain might look like\n> \n> -> link_alt_odb_entries\n> -> link_alt_odb_entry\n> -> read_info_alternates\n> -> link_alt_odb_entries\n> \n> That is why we need to convert all these functions at the same time.\n> \n> Signed-off-by: Jonathan Nieder \n> Signed-off-by: Stefan Beller \n> ---\n\nLooks good.\n\n> object-store.h | 4 ++++\n> sha1_file.c | 36 ++++++++++++++++--------------------\n> 2 files changed, 20 insertions(+), 20 deletions(-)\n> \n> diff --git a/object-store.h b/object-store.h\n> index 514ad94287..f283fbdba9 100644\n> --- a/object-store.h\n> +++ b/object-store.h\n> @@ -18,6 +18,10 @@ struct alternate_object_database {\n> \tchar loose_objects_subdir_seen[256];\n> \tstruct oid_array loose_objects_cache;\n> \n> +\t/*\n> +\t * Path to the alternative object store. If this is a relative path,\n> +\t * it is relative to the current working directory.\n> +\t */\n\nThanks for adding the comment.\n\n> \tchar path[FLEX_ARRAY];\n> };\n> #define prepare_alt_odb(r) prepare_alt_odb_##r()\n> diff --git a/sha1_file.c b/sha1_file.c\n> index c26921df4a..6e5105a252 100644\n> --- a/sha1_file.c\n> +++ b/sha1_file.c\n> @@ -390,11 +390,10 @@ static int alt_odb_usable(struct raw_object_store *o,\n> * SHA1, an extra slash for the first level indirection, and the\n> * terminating NUL.\n> */\n> -#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)\n> -static void read_info_alternates_the_repository(const char *relative_base,\n> -\t\t\t\t\t\tint depth);\n> -#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)\n> -static int link_alt_odb_entry_the_repository(const char *entry,\n> +static void read_info_alternates(struct repository *r,\n> +\t\t\t\t const char *relative_base,\n> +\t\t\t\t int depth);\n> +static int link_alt_odb_entry(struct repository *r, const char *entry,\n> \tconst char *relative_base, int depth, const char *normalized_objdir)\n> {\n> \tstruct alternate_object_database *ent;\n> @@ -420,7 +419,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,\n> \twhile (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')\n> \t\tstrbuf_setlen(&pathbuf, pathbuf.len - 1);\n> \n> -\tif (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {\n> +\tif (!alt_odb_usable(&r->objects, &pathbuf, normalized_objdir)) {\n> \t\tstrbuf_release(&pathbuf);\n> \t\treturn -1;\n> \t}\n> @@ -428,12 +427,12 @@ static int link_alt_odb_entry_the_repository(const char *entry,\n> \tent = alloc_alt_odb(pathbuf.buf);\n> \n> \t/* add the alternate entry */\n> -\t*the_repository->objects.alt_odb_tail = ent;\n> -\tthe_repository->objects.alt_odb_tail = &(ent->next);\n> +\t*r->objects.alt_odb_tail = ent;\n> +\tr->objects.alt_odb_tail = &(ent->next);\n> \tent->next = NULL;\n> \n> \t/* recursively add alternates */\n> -\tread_info_alternates(the_repository, pathbuf.buf, depth + 1);\n> +\tread_info_alternates(r, pathbuf.buf, depth + 1);\n> \n> \tstrbuf_release(&pathbuf);\n> \treturn 0;\n> @@ -468,12 +467,8 @@ static const char *parse_alt_odb_entry(const char *string,\n> \treturn end;\n> }\n> \n> -#define link_alt_odb_entries(r, a, s, rb, d) \\\n> -\tlink_alt_odb_entries_##r(a, s, rb, d)\n> -static void link_alt_odb_entries_the_repository(const char *alt,\n> -\t\t\t\t\t\tint sep,\n> -\t\t\t\t\t\tconst char *relative_base,\n> -\t\t\t\t\t\tint depth)\n> +static void link_alt_odb_entries(struct repository *r, const char *alt,\n> +\t\t\t\t int sep, const char *relative_base, int depth)\n> {\n> \tstruct strbuf objdirbuf = STRBUF_INIT;\n> \tstruct strbuf entry = STRBUF_INIT;\n> @@ -487,7 +482,7 @@ static void link_alt_odb_entries_the_repository(const char *alt,\n> \t\treturn;\n> \t}\n> \n> -\tstrbuf_add_absolute_path(&objdirbuf, get_object_directory());\n> +\tstrbuf_add_absolute_path(&objdirbuf, r->objects.objectdir);\n> \tif (strbuf_normalize_path(&objdirbuf) < 0)\n> \t\tdie(\"unable to normalize object directory: %s\",\n> \t\t objdirbuf.buf);\n> @@ -496,15 +491,16 @@ static void link_alt_odb_entries_the_repository(const char *alt,\n> \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n> \t\tif (!entry.len)\n> \t\t\tcontinue;\n> -\t\tlink_alt_odb_entry(the_repository, entry.buf,\n> +\t\tlink_alt_odb_entry(r, entry.buf,\n> \t\t\t\t relative_base, depth, objdirbuf.buf);\n> \t}\n> \tstrbuf_release(&entry);\n> \tstrbuf_release(&objdirbuf);\n> }\n> \n> -static void read_info_alternates_the_repository(const char *relative_base,\n> -\t\t\t\t\t\tint depth)\n> +static void read_info_alternates(struct repository *r,\n> +\t\t\t\t const char *relative_base,\n> +\t\t\t\t int depth)\n> {\n> \tchar *path;\n> \tstruct strbuf buf = STRBUF_INIT;\n> @@ -516,7 +512,7 @@ static void read_info_alternates_the_repository(const char *relative_base,\n> \t\treturn;\n> \t}\n> \n> -\tlink_alt_odb_entries(the_repository, buf.buf, '\\n', relative_base, depth);\n> +\tlink_alt_odb_entries(r, buf.buf, '\\n', relative_base, depth);\n> \tstrbuf_release(&buf);\n> \tfree(path);\n> }\n> -- \n> 2.16.1.291.g4437f3f132-goog\n> \n\n-- \nBrandon Williams\n"},{"id":"339893","messageId":"20180222005149.GH127348@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-17-sbeller@google.com","subject":"Re: [PATCH 16/27] sha1_file: add repository argument to sha1_file_name","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-22T00:51:49Z","receivedAt":"2018-02-22T00:51:57Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/20, Stefan Beller wrote:\n> Add a repository argument to allow sha1_file_name callers to be more\n> specific about which repository to handle. This is a small mechanical\n> change; it doesn't change the implementation to handle repositories\n> other than the_repository yet.\n> \n> As with the previous commits, use a macro to catch callers passing a\n> repository other than the_repository at compile time.\n> \n> While at it, move the declaration to object-store.h, where it should\n> be easier to find.\n\nSeems like we may want to make a sha1-file.h or an oid-file.h or\nsomething like that at some point as that seems like a better place for\nthe function than in the object-store.h file?\n\n\n-- \nBrandon Williams\n"},{"id":"339902","messageId":"20180222062635.GC133592@aiede.svl.corp.google.com","threadId":"47823","inReplyTo":"20180221015430.96054-2-sbeller@google.com","subject":"Re: [PATCH 01/27] repository: introduce raw object store field","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2018-02-22T06:26:35Z","receivedAt":"2018-02-22T06:26:46Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nStefan Beller wrote:\n\n> The raw object store field will contain any objects needed for\n> access to objects in a given repository.\n>\n> This patch introduces the raw object store and populates it with the\n> `objectdir`, which used to be part of the repository struct.\n>\n> As the struct gains members, we'll also populate the function to clear\n> the memory for these members.\n>\n> In a later we'll introduce a struct object_parser, that will complement\n> the object parsing in a repository struct: The raw object parser is the\n> layer that will provide access to raw object content, while the higher\n> level object parser code will parse raw objects and keeps track of\n> parenthood and other object relationships using 'struct object'.\n> For now only add the lower level to the repository struct.\n>\n> Signed-off-by: Stefan Beller \n> Signed-off-by: Jonathan Nieder \n> ---\n\nHeh, I suppose that sign-off makes me not a great candidate for\nreviewing this. It's been long enough since I looked at the patch\nthat I feel okay trying anyway.\n\n[...]\n> --- /dev/null\n> +++ b/object-store.h\n> @@ -0,0 +1,15 @@\n> +#ifndef OBJECT_STORE_H\n> +#define OBJECT_STORE_H\n> +\n> +struct raw_object_store {\n> +\t/*\n> +\t * Path to the repository's object store.\n> +\t * Cannot be NULL after initialization.\n> +\t */\n> +\tchar *objectdir;\n> +};\n> +#define RAW_OBJECT_STORE_INIT { NULL }\n\nWho owns and is responsible for freeing objectdir?\n\n> +\n> +void raw_object_store_clear(struct raw_object_store *o);\n\nOh, that answers that.\n\nIt could be worth a note in the doc comment anyway, but I don't mind\nnot having one.\n\n[...]\n> +\n> +void raw_object_store_clear(struct raw_object_store *o)\n> +{\n> +\tfree(o->objectdir);\n> +}\n\nShould this use FREE_AND_NULL?\n\n[...]\n> --- a/repository.c\n> +++ b/repository.c\n[...]\n> @@ -42,8 +49,8 @@ static void repo_setup_env(struct repository *repo)\n> \t\t\t\t\t\t !repo->ignore_env);\n> \tfree(repo->commondir);\n> \trepo->commondir = strbuf_detach(&sb, NULL);\n> -\tfree(repo->objectdir);\n> -\trepo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,\n> +\tfree(repo->objects.objectdir);\n\nShould this call raw_object_store_clear instead of calling free\ndirectly?\n\n> +\trepo->objects.objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,\n> \t\t\t\t\t \"objects\", !repo->ignore_env);\n\nLong line. One way to break it would be\n\n\trepo->objects.objectdir =\n\t\tgit_path_from_env(DB_ENVIRONMENT, repo->commondir,\n\t\t\t\t \"objects\", !repo->ignore_env);\n\n> \tfree(repo->graft_file);\n> \trepo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,\n> @@ -209,12 +216,14 @@ void repo_clear(struct repository *repo)\n> {\n> \tFREE_AND_NULL(repo->gitdir);\n> \tFREE_AND_NULL(repo->commondir);\n> -\tFREE_AND_NULL(repo->objectdir);\n> \tFREE_AND_NULL(repo->graft_file);\n> \tFREE_AND_NULL(repo->index_file);\n> \tFREE_AND_NULL(repo->worktree);\n> \tFREE_AND_NULL(repo->submodule_prefix);\n> \n> +\traw_object_store_clear(&repo->objects);\n> +\tmemset(&repo->objects, 0, sizeof(repo->objects));\n\nIf raw_object_store_clear uses FREE_AND_NULL, this memset won't be necessary.\n\n[...]\n> --- a/repository.h\n> +++ b/repository.h\n> @@ -1,6 +1,8 @@\n> #ifndef REPOSITORY_H\n> #define REPOSITORY_H\n> \n> +#include \"object-store.h\"\n> +\n> struct config_set;\n> struct index_state;\n> struct submodule_cache;\n> @@ -21,10 +23,9 @@ struct repository {\n> \tchar *commondir;\n> \n> \t/*\n> -\t * Path to the repository's object store.\n> -\t * Cannot be NULL after initialization.\n> +\t * Holds any information related to the object store.\n> \t */\n\nThis comment doesn't make it clear to me what the field represents.\nCan it be replaced with some of the description from the commit\nmessage?\n\n> -\tchar *objectdir;\n> +\tstruct raw_object_store objects;\n> \n\nThanks,\nJonathan\n"},{"id":"339903","messageId":"20180222064423.GD133592@aiede.svl.corp.google.com","threadId":"47823","inReplyTo":"20180221015430.96054-6-sbeller@google.com","subject":"Re: [PATCH 05/27] object-store: move packed_git and packed_git_mru to object store","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2018-02-22T06:44:23Z","receivedAt":"2018-02-22T06:47:55Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nStefan Beller wrote:\n\n> In a process with multiple repositories open, packfile accessors\n> should be associated to a single repository and not shared globally.\n> Move packed_git and packed_git_mru into the_repository and adjust\n> callers to reflect this.\n>\n> Patch generated by\n>\n> 1. Moving the struct packed_git declaration to object-store.h\n> and packed_git, packed_git_mru globals to struct object_store.\n>\n> 2. Applying the semantic patch\n> contrib/coccinelle/refactoring/packed_git.cocci to adjust callers.\n> This semantic patch is placed in a sub directory of the coccinelle\n> contrib dir, as this semantic patch is not expected to be of general\n> usefulness; it is only useful during developing this series and\n> merging it with other topics in flight. At a later date, just\n> delete that semantic patch.\n\nCan the semantic patch go in the commit message instead? It is very\nbrief.\n\nActually, I don't see this semantic patch in the diffstat. Is the\ncommit message stale?\n\n> 3. Applying line wrapping fixes from \"make style\" to break the\n> resulting long lines.\n>\n> 4. Adding missing #includes of repository.h and object-store.h\n> where needed.\n\nIs there a way to automate this step? (I'm asking for my own\nreference when writing future patches, not because of any concern\nabout the correctness of this one.)\n>\n> 5. As the packfiles are now owned by an objectstore/repository, which\n> is ephemeral unlike globals, we introduce memory leaks. So address\n> them in raw_object_store_clear().\n\nThe compound words are confusing me. What is an\nobjectstore/repository? Are these referring to particular identifiers\nor something else?\n\nWould some wording like the following work?\n\n 5. Freeing packed_git and packed_git_mru in raw_object_store_clear\n to avoid a per-repository memory leak. Previously they were\n global singletons, so code to free them did not exist.\n\n[...]\n> --- a/builtin/index-pack.c\n> +++ b/builtin/index-pack.c\n> @@ -12,6 +12,7 @@\n> #include \"exec_cmd.h\"\n> #include \"streaming.h\"\n> #include \"thread-utils.h\"\n> +#include \"object-store.h\"\n> #include \"packfile.h\"\n> \n> static const char index_pack_usage[] =\n\nChange from a different patch leaked into this one?\n\n[...]\n> +++ b/builtin/pack-objects.c\n[...]\n> @@ -1044,7 +1045,7 @@ static int want_object_in_pack(const struct object_id *oid,\n> \t\t\t}\n> \t\t\twant = want_found_object(exclude, p);\n> \t\t\tif (!exclude && want > 0)\n> -\t\t\t\tlist_move(&p->mru, &packed_git_mru);\n> +\t\t\t\tlist_move(&p->mru, &the_repository->objects.packed_git_mru);\n\nLong line. Can \"make style\" catch this?\n\n[...]\n> +++ b/builtin/receive-pack.c\n> @@ -7,6 +7,7 @@\n> #include \"sideband.h\"\n> #include \"run-command.h\"\n> #include \"exec_cmd.h\"\n> +#include \"object-store.h\"\n> #include \"commit.h\"\n> #include \"object.h\"\n> #include \"remote.h\"\n\nAnother change leaked in?\n\n[...]\n> --- a/cache.h\n> +++ b/cache.h\n> @@ -1585,35 +1585,6 @@ struct pack_window {\n> \tunsigned int inuse_cnt;\n> };\n> \n> -extern struct packed_git {\n[...]\n> -} *packed_git;\n\nMove detecting diff confirms that this wasn't modified. Thanks for\ncreating it.\n\n[...]\n> +++ b/fast-import.c\n[...]\n> @@ -1110,7 +1112,7 @@ static int store_object(\n> \tif (e->idx.offset) {\n> \t\tduplicate_count_by_type[type]++;\n> \t\treturn 1;\n> -\t} else if (find_sha1_pack(oid.hash, packed_git)) {\n> +\t} else if (find_sha1_pack(oid.hash, the_repository->objects.packed_git)) {\n\nLong line. (I'll refrain from commenting about any further ones.)\n\n[...]\n> +++ b/http-push.c\n> @@ -1,4 +1,5 @@\n> #include \"cache.h\"\n> +#include \"object-store.h\"\n> #include \"commit.h\"\n> #include \"tag.h\"\n> #include \"blob.h\"\n\nStray change?\n\n> diff --git a/http-walker.c b/http-walker.c\n> index 07c2b1af82..8bb5d991bb 100644\n> --- a/http-walker.c\n> +++ b/http-walker.c\n> @@ -4,6 +4,7 @@\n> #include \"http.h\"\n> #include \"list.h\"\n> #include \"transport.h\"\n> +#include \"object-store.h\"\n> #include \"packfile.h\"\n> \n> struct alt_base {\n\nSame question.\n\n> diff --git a/http.c b/http.c\n> index 31755023a4..a4a9e583c7 100644\n> --- a/http.c\n> +++ b/http.c\n> @@ -1,6 +1,7 @@\n> #include \"git-compat-util.h\"\n> #include \"http.h\"\n> #include \"config.h\"\n> +#include \"object-store.h\"\n> #include \"pack.h\"\n> #include \"sideband.h\"\n> #include \"run-command.h\"\n\nLikewise.\n\n> diff --git a/object-store.h b/object-store.h\n> index e78eea1dde..1de9e07102 100644\n> --- a/object-store.h\n> +++ b/object-store.h\n> @@ -52,6 +52,30 @@ void add_to_alternates_memory(const char *dir);\n> */\n> struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);\n> \n> +struct packed_git {\n> +\tstruct packed_git *next;\n> +\tstruct list_head mru;\n> +\tstruct pack_window *windows;\n> +\toff_t pack_size;\n> +\tconst void *index_data;\n> +\tsize_t index_size;\n> +\tuint32_t num_objects;\n> +\tuint32_t num_bad_objects;\n> +\tunsigned char *bad_object_sha1;\n> +\tint index_version;\n> +\ttime_t mtime;\n> +\tint pack_fd;\n> +\tunsigned pack_local:1,\n> +\t\t pack_keep:1,\n> +\t\t freshened:1,\n> +\t\t do_not_close:1,\n> +\t\t pack_promisor:1;\n> +\tunsigned char sha1[20];\n> +\tstruct revindex_entry *revindex;\n> +\t/* something like \".git/objects/pack/xxxxx.pack\" */\n> +\tchar pack_name[FLEX_ARRAY]; /* more */\n> +};\n> +\n> struct raw_object_store {\n> \t/*\n> \t * Path to the repository's object store.\n> @@ -59,10 +83,25 @@ struct raw_object_store {\n> \t */\n> \tchar *objectdir;\n> \n> +\tstruct packed_git *packed_git;\n> +\t/*\n> +\t * A most-recently-used ordered version of the packed_git list, which can\n> +\t * be iterated instead of packed_git (and marked via mru_mark).\n> +\t */\n> +\tstruct list_head packed_git_mru;\n\nI don't understand the new part of the comment. Can you explain here,\nfor me?\n\nIs this meant as a list of related functions, an explanation of what the\nfield is for, or something else?\n\n> +\n> \tstruct alternate_object_database *alt_odb_list;\n> \tstruct alternate_object_database **alt_odb_tail;\n> };\n> -#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }\n> +\n> +/*\n> + * The mru list_head is supposed to be initialized using\n> + * the LIST_HEAD macro, assigning prev/next to itself.\n> + * However this doesn't work in this case as some compilers dislike\n> + * that macro on member variables. Use NULL instead as that is defined\n> + * and accepted, deferring the real init to prepare_packed_git_mru(). */\n\nstyle nit: '*/' should be on its own line.\n\nMore importantly, we can avoid such an issue as described by Junio. :)\n\n> +#define __MRU_INIT { NULL, NULL }\n\nIdentifiers with leading underscores like this are in a reserved\nnamespace for the language implementation --- we can't count on them\nbeing available for our own use.\n\n> +#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL }\n[...]\n> --- a/object.c\n> +++ b/object.c\n> @@ -466,4 +466,11 @@ void raw_object_store_clear(struct raw_object_store *o)\n> \n> \tfree_alt_odbs(o);\n> \to->alt_odb_tail = NULL;\n> +\n> +\twhile (!list_empty(&o->packed_git_mru))\n> +\t\tlist_del(&o->packed_git_mru);\n> +\t/*\n> +\t * TODO: call close_all_packs once migrated to\n> +\t * take an object store argument\n> +\t */\n\nCan you say more about this TODO? Does this mean the patches are out\nof order (i.e. that raw_object_store_clear leaves behind a leak until\na later patch)?\n\n[...]\n> --- a/pack-check.c\n> +++ b/pack-check.c\n> @@ -1,5 +1,6 @@\n> #include \"cache.h\"\n> #include \"pack.h\"\n> +#include \"object-store.h\"\n> #include \"pack-revindex.h\"\n> #include \"progress.h\"\n> #include \"packfile.h\"\n\nAnother unexplained #include (I'll refrain from pointing out later\nones).\n\nThe rest looks good.\n\nThanks,\nJonathan\n"},{"id":"339905","messageId":"20180222065122.GE133592@aiede.svl.corp.google.com","threadId":"47823","inReplyTo":"20180221015430.96054-14-sbeller@google.com","subject":"Re: [PATCH 13/27] sha1_file: add repository argument to prepare_alt_odb","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2018-02-22T06:51:22Z","receivedAt":"2018-02-22T06:51:29Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nStefan Beller wrote:\n\n> See previous patch for explanation.\n>\n> While at it, move the declaration to object-store.h,\n> where it should be easier to find.\n\nWhich declaration? It looks like prepare_alt_odb is already in\nobject-store.h.\n\n[...]\n> --- a/builtin/fsck.c\n> +++ b/builtin/fsck.c\n> @@ -717,7 +717,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n> \t} else {\n> \t\tfsck_object_dir(get_object_directory());\n> \n> -\t\tprepare_alt_odb();\n> +\t\tprepare_alt_odb(the_repository);\n\nPatch 2 added a #include of \"repository.h\". Good.\n\n(I checked because with the definition of prepare_alt_odb as a macro,\nthe function call would compile correctly even if the_repository\nweren't in scope, but we want to include what we use as a matter of\nstyle/maintainability.)\n\n[...]\n> --- a/packfile.c\n> +++ b/packfile.c\n> @@ -890,7 +890,7 @@ void prepare_packed_git(void)\n> \tif (the_repository->objects.packed_git_initialized)\n> \t\treturn;\n> \tprepare_packed_git_one(get_object_directory(), 1);\n> -\tprepare_alt_odb();\n> +\tprepare_alt_odb(the_repository);\n\nAlso good, since patch 3 added a #include of \"repository.h\".\n\n[...]\n> --- a/sha1_file.c\n> +++ b/sha1_file.c\n> @@ -23,6 +23,7 @@\n> #include \"sha1-lookup.h\"\n> #include \"bulk-checkin.h\"\n> #include \"repository.h\"\n> +#include \"object-store.h\"\n\nShould this #include have been added in an earlier patch, since the\nfile both uses and defines prepare_alt_odb, which is declared there?\n\nThe rest looks good.\n\nThanks,\nJonathan\n"},{"id":"339906","messageId":"20180222065221.GF133592@aiede.svl.corp.google.com","threadId":"47823","inReplyTo":"20180221015430.96054-18-sbeller@google.com","subject":"Re: [PATCH 17/27] sha1_file: add repository argument to stat_sha1_file","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2018-02-22T06:52:21Z","receivedAt":"2018-02-22T06:52:29Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nStefan Beller wrote:\n\n> Add a repository argument to allow the stat_sha1_file caller to be\n> more specific about which repository to act on. This is a small\n> mechanical change; it doesn't change the implementation to handle\n> repositories other than the_repository yet.\n>\n> As with the previous commits, use a macro to catch callers passing a\n> repository other than the_repository at compile time.\n>\n> Signed-off-by: Stefan Beller \n> Signed-off-by: Jonathan Nieder \n> ---\n> sha1_file.c | 9 +++++----\n> 1 file changed, 5 insertions(+), 4 deletions(-)\n\nSimple and obviously correct.\nReviewed-by: Jonathan Nieder \n"},{"id":"339907","messageId":"20180222065321.GG133592@aiede.svl.corp.google.com","threadId":"47823","inReplyTo":"20180221015430.96054-22-sbeller@google.com","subject":"Re: [PATCH 21/27] sha1_file: add repository argument to sha1_loose_object_info","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2018-02-22T06:53:22Z","receivedAt":"2018-02-22T06:53:30Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nStefan Beller wrote:\n\n> Add a repository argument to allow the sha1_loose_object_info caller\n> to be more specific about which repository to act on. This is a small\n> mechanical change; it doesn't change the implementation to handle\n> repositories other than the_repository yet.\n>\n> As with the previous commits, use a macro to catch callers passing a\n> repository other than the_repository at compile time.\n>\n> Signed-off-by: Stefan Beller \n> Signed-off-by: Jonathan Nieder \n> ---\n> sha1_file.c | 9 +++++----\n> 1 file changed, 5 insertions(+), 4 deletions(-)\n\nSimple and obviously correct.\nReviewed-by: Jonathan Nieder \n"},{"id":"339908","messageId":"20180222065450.GH133592@aiede.svl.corp.google.com","threadId":"47823","inReplyTo":"20180221015430.96054-27-sbeller@google.com","subject":"Re: [PATCH 26/27] sha1_file: allow map_sha1_file to handle arbitrary repositories","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2018-02-22T06:54:50Z","receivedAt":"2018-02-22T06:54:59Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Stefan Beller wrote:\n\n> Signed-off-by: Stefan Beller \n> Signed-off-by: Jonathan Nieder \n> ---\n> object-store.h | 3 +--\n> sha1_file.c | 5 +++--\n> 2 files changed, 4 insertions(+), 4 deletions(-)\n\nThanks: this is short and simple, making my life as a reviewer very\neasy.\n\nReviewed-by: Jonathan Nieder \n"},{"id":"340003","messageId":"20180223095640.25876-1-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180214180814.GA139458@google.com","subject":"[PATCH 0/2] Fix initializing the_hash_algo","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-23T09:56:38Z","receivedAt":"2018-02-23T09:57:11Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Thu, Feb 15, 2018 at 1:08 AM, Brandon Williams wrote:\n> At some point yes we would definitely want the setup code to fully\n> initialize a repository object (in this case the_repository).  And I\n> would even like to change the function signatures of all the builtin\n> commands to take a repository object so that they don't implicitly rely\n> on the_repository at all.\n>\n> When I introduced struct repository I seem to remember there being a\n> couple things which were different about setup that made it difficult to\n> simply call repo_init() on the_repository.  If you can fix whatever\n> those issues with setup were (I can't remember all of them) then that\n> would be great :)\n\nI can certainly try! I start to remember all the hairy details in that\nsetup code.\n\nThe first step may be something like this, which identifies all the\n\"repo init\" entry points. This is basically a revert of e26f7f19b6\n(repository: pre-initialize hash algo pointer - 2018-01-19) and doing\nthings the proper way, hopefully.\n\nThis is on 'master', independent from Stefan's series. I have another\npatch on top of that series to remove the use of ignore_env in\nsha1_file.c (and things seem to work). Basically whenever you have to\ninitialize the hash algorithm, there's a good chance you need to\ninitialize object store as well. But I'll hold that off until\nStefan's and this one are both merged.\n\nBut yeah, it looks like we need some surgery in setup.c if we want\nsomething as pretty as repo_submodule_init() but for the main repo.\n\nNguyễn Thái Ngọc Duy (2):\n setup.c: initialize the_repository correctly in all cases\n Revert \"repository: pre-initialize hash algo pointer\"\n\n builtin/index-pack.c | 5 +++++\n builtin/init-db.c | 3 ++-\n cache.h | 3 ++-\n common-main.c | 4 ++++\n diff-no-index.c | 5 +++++\n path.c | 2 +-\n repository.c | 2 +-\n setup.c | 5 ++++-\n t/helper/test-dump-split-index.c | 2 ++\n 9 files changed, 26 insertions(+), 5 deletions(-)\n\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340004","messageId":"20180223095640.25876-2-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180223095640.25876-1-pclouds@gmail.com","subject":"[PATCH 1/2] setup.c: initialize the_repository correctly in all cases","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-23T09:56:39Z","receivedAt":"2018-02-23T09:57:13Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"There are many ways for any command to access a git repository:\n\n- most of them will try to discover the .git dir via\n setup_git_directory() and friends\n\n- the server side programs already know where the repo is and prepare\n with enter_repo()\n\n- special commands that deal with repo creation (init/clone) use\n init_db() once the new repo is ready for access.\n\n- somebody accesses $GIT_DIR before any of above functions are called\n and accidentally sets up a git repository by set_git_dir() alone\n\n\"the_repository\" is partially set up via set_git_dir() at some point\nin all four cases. The hash algorithm though is configured later after\n.git/config is read.\n\nSo far proper repo initialization is done only for the first case [1].\nThe second case is not covered (but that's fine [3]). The third case\nwas found and worked around in [2]. The fourth case is a buggy one,\nwhich should be fixed already by jk/no-looking-at-dotgit-outside-repo\nand never happens again.\n\nThis patch makes sure all cases initialize the hash algorithm in\nthe_repository correctly. Both second and third cases must run\ncheck_repo_format() before \"entering\" it. Eventually we probably just\nrename this function to init_repo() or something.\n\n[1] 78a6766802 (Integrate hash algorithm support with repo setup -\n 2017-11-12)\n\n[2] e26f7f19b6 (repository: pre-initialize hash algo pointer -\n 2018-01-19)\n\n[3] the reason server side is still running ok with no hash algo\n before [2] is because the programs that use enter_repo() do very\n little then spawn a new program (like pack-objects or\n upload-archive) to do the heavy lifting. These programs already\n use setup_git_dir..()\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/init-db.c | 3 ++-\n cache.h | 3 ++-\n path.c | 2 +-\n setup.c | 5 ++++-\n 4 files changed, 9 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex c9b7946bad..d119d9906b 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -9,6 +9,7 @@\n #include \"builtin.h\"\n #include \"exec_cmd.h\"\n #include \"parse-options.h\"\n+#include \"repository.h\"\n \n #ifndef DEFAULT_GIT_TEMPLATE_DIR\n #define DEFAULT_GIT_TEMPLATE_DIR \"/usr/share/git-core/templates\"\n@@ -369,7 +370,7 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \t * config file, so this will not fail. What we are catching\n \t * is an attempt to reinitialize new repository with an old tool.\n \t */\n-\tcheck_repository_format();\n+\tcheck_repository_format(the_repository);\n \n \treinit = create_default_files(template_dir, original_git_dir);\n \ndiff --git a/cache.h b/cache.h\nindex 21fbcc2414..6b97138264 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -894,6 +894,7 @@ extern int repository_format_precious_objects;\n extern char *repository_format_partial_clone;\n extern const char *core_partial_clone_filter_default;\n \n+struct repository;\n struct repository_format {\n \tint version;\n \tint precious_objects;\n@@ -926,7 +927,7 @@ int verify_repository_format(const struct repository_format *format,\n * set_git_dir() before calling this, and use it only for \"are we in a valid\n * repo?\".\n */\n-extern void check_repository_format(void);\n+extern void check_repository_format(struct repository *);\n \n #define MTIME_CHANGED\t0x0001\n #define CTIME_CHANGED\t0x0002\ndiff --git a/path.c b/path.c\nindex da8b655730..a544252198 100644\n--- a/path.c\n+++ b/path.c\n@@ -827,7 +827,7 @@ const char *enter_repo(const char *path, int strict)\n \n \tif (is_git_directory(\".\")) {\n \t\tset_git_dir(\".\");\n-\t\tcheck_repository_format();\n+\t\tcheck_repository_format(the_repository);\n \t\treturn path;\n \t}\n \ndiff --git a/setup.c b/setup.c\nindex c5d55dcee4..a82103832e 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -1180,11 +1180,14 @@ int git_config_perm(const char *var, const char *value)\n \treturn -(i & 0666);\n }\n \n-void check_repository_format(void)\n+/* optionally configure \"repo\" to the correct format */\n+void check_repository_format(struct repository *repo)\n {\n \tstruct repository_format repo_fmt;\n \tcheck_repository_format_gently(get_git_dir(), &repo_fmt, NULL);\n \tstartup_info->have_repository = 1;\n+\tif (repo)\n+\t\trepo_set_hash_algo(repo, repo_fmt.hash_algo);\n }\n \n /*\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340005","messageId":"20180223095640.25876-3-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180223095640.25876-1-pclouds@gmail.com","subject":"[PATCH 2/2] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-23T09:56:40Z","receivedAt":"2018-02-23T09:57:18Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This reverts commit e26f7f19b6c7485f04234946a59ab8f4fd21d6d1. The root\nproblem, git clone not setting up the_hash_algo, has been fixed in the\nprevious patch.\n\nAs a result of the revert, some code paths that use the_hash_algo\nwithout initialization is revealed and fixed here. It's basically\ncommands that are allowed to run without a repository. The fix here is\nnot the best. We probably could figure out the hash algorithm from input\nsomehow.\n\nSince this is a dangerous move and could potentially break stuff after\nrelease (and leads to workaround like the reverted commit), the\nworkaround technically remains, but is hidden behind a new environment\nvariable GIT_HASH_FIXUP. This should let the users continue to use git\nwhile we fix the problem.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/index-pack.c | 5 +++++\n common-main.c | 4 ++++\n diff-no-index.c | 5 +++++\n repository.c | 2 +-\n t/helper/test-dump-split-index.c | 2 ++\n 5 files changed, 17 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 7e3e1a461c..8ee935504e 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1673,6 +1673,11 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n \tif (prefix && chdir(prefix))\n \t\tdie(_(\"Cannot come back to cwd\"));\n \n+\tif (!the_hash_algo) {\n+\t\twarning(_(\"Running without a repository, assuming SHA-1 hash\"));\n+\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n+\t}\n+\n \tfor (i = 1; i < argc; i++) {\n \t\tconst char *arg = argv[i];\n \ndiff --git a/common-main.c b/common-main.c\nindex 6a689007e7..12aec36794 100644\n--- a/common-main.c\n+++ b/common-main.c\n@@ -1,6 +1,7 @@\n #include \"cache.h\"\n #include \"exec_cmd.h\"\n #include \"attr.h\"\n+#include \"repository.h\"\n \n /*\n * Many parts of Git have subprograms communicate via pipe, expect the\n@@ -40,5 +41,8 @@ int main(int argc, const char **argv)\n \n \trestore_sigpipe_to_default();\n \n+\tif (getenv(\"GIT_HASH_FIXUP\"))\n+\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n+\n \treturn cmd_main(argc, argv);\n }\ndiff --git a/diff-no-index.c b/diff-no-index.c\nindex 0ed5f0f496..f038f665bc 100644\n--- a/diff-no-index.c\n+++ b/diff-no-index.c\n@@ -241,6 +241,11 @@ void diff_no_index(struct rev_info *revs,\n \tstruct strbuf replacement = STRBUF_INIT;\n \tconst char *prefix = revs->prefix;\n \n+\tif (!the_hash_algo) {\n+\t\twarning(_(\"Running without a repository, assuming SHA-1 hash\"));\n+\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n+\t}\n+\n \tdiff_setup(&revs->diffopt);\n \tfor (i = 1; i < argc - 2; ) {\n \t\tint j;\ndiff --git a/repository.c b/repository.c\nindex 4ffbe9bc94..0d715f4fdb 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -5,7 +5,7 @@\n \n /* The main repository */\n static struct repository the_repo = {\n-\tNULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0\n+\tNULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, NULL, 0, 0\n };\n struct repository *the_repository = &the_repo;\n \ndiff --git a/t/helper/test-dump-split-index.c b/t/helper/test-dump-split-index.c\nindex e44430b699..b759fe3fc1 100644\n--- a/t/helper/test-dump-split-index.c\n+++ b/t/helper/test-dump-split-index.c\n@@ -12,6 +12,8 @@ int cmd_main(int ac, const char **av)\n \tstruct split_index *si;\n \tint i;\n \n+\tsetup_git_directory();\n+\n \tdo_read_index(&the_index, av[1], 1);\n \tprintf(\"own %s\\n\", sha1_to_hex(the_index.sha1));\n \tsi = the_index.split_index;\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340052","messageId":"CAGZ79karK7MT_qpFyYfVkioeBsBHidGdC_QfisPmG1a4dQuk5A@mail.gmail.com","threadId":"47823","inReplyTo":"20180223095640.25876-3-pclouds@gmail.com","subject":"Re: [PATCH 2/2] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-23T18:24:39Z","receivedAt":"2018-02-23T18:34:05Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Feb 23, 2018 at 1:56 AM, Nguyễn Thái Ngọc Duy wrote:\n> This reverts commit e26f7f19b6c7485f04234946a59ab8f4fd21d6d1. The root\n> problem, git clone not setting up the_hash_algo, has been fixed in the\n> previous patch.\n>\n> As a result of the revert, some code paths that use the_hash_algo\n> without initialization is revealed and fixed here. It's basically\n> commands that are allowed to run without a repository. The fix here is\n> not the best. We probably could figure out the hash algorithm from input\n> somehow.\n>\n> Since this is a dangerous move and could potentially break stuff after\n> release (and leads to workaround like the reverted commit), the\n> workaround technically remains, but is hidden behind a new environment\n> variable GIT_HASH_FIXUP. This should let the users continue to use git\n> while we fix the problem.\n>\n> Signed-off-by: Nguyễn Thái Ngọc Duy \n> ---\n> builtin/index-pack.c | 5 +++++\n> common-main.c | 4 ++++\n> diff-no-index.c | 5 +++++\n> repository.c | 2 +-\n> t/helper/test-dump-split-index.c | 2 ++\n> 5 files changed, 17 insertions(+), 1 deletion(-)\n>\n> diff --git a/builtin/index-pack.c b/builtin/index-pack.c\n> index 7e3e1a461c..8ee935504e 100644\n> --- a/builtin/index-pack.c\n> +++ b/builtin/index-pack.c\n> @@ -1673,6 +1673,11 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n> if (prefix && chdir(prefix))\n> die(_(\"Cannot come back to cwd\"));\n>\n> + if (!the_hash_algo) {\n> + warning(_(\"Running without a repository, assuming SHA-1 hash\"));\n> + repo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n> + }\n> +\n> for (i = 1; i < argc; i++) {\n> const char *arg = argv[i];\n>\n> diff --git a/common-main.c b/common-main.c\n> index 6a689007e7..12aec36794 100644\n> --- a/common-main.c\n> +++ b/common-main.c\n> @@ -1,6 +1,7 @@\n> #include \"cache.h\"\n> #include \"exec_cmd.h\"\n> #include \"attr.h\"\n> +#include \"repository.h\"\n>\n> /*\n> * Many parts of Git have subprograms communicate via pipe, expect the\n> @@ -40,5 +41,8 @@ int main(int argc, const char **argv)\n>\n> restore_sigpipe_to_default();\n>\n> + if (getenv(\"GIT_HASH_FIXUP\"))\n> + repo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n> +\n> return cmd_main(argc, argv);\n> }\n> diff --git a/diff-no-index.c b/diff-no-index.c\n> index 0ed5f0f496..f038f665bc 100644\n> --- a/diff-no-index.c\n> +++ b/diff-no-index.c\n> @@ -241,6 +241,11 @@ void diff_no_index(struct rev_info *revs,\n> struct strbuf replacement = STRBUF_INIT;\n> const char *prefix = revs->prefix;\n>\n> + if (!the_hash_algo) {\n> + warning(_(\"Running without a repository, assuming SHA-1 hash\"));\n> + repo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n> + }\n> +\n> diff_setup(&revs->diffopt);\n> for (i = 1; i < argc - 2; ) {\n> int j;\n> diff --git a/repository.c b/repository.c\n> index 4ffbe9bc94..0d715f4fdb 100644\n> --- a/repository.c\n> +++ b/repository.c\n> @@ -5,7 +5,7 @@\n>\n> /* The main repository */\n> static struct repository the_repo = {\n> - NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0\n> + NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, NULL, 0, 0\n\nI am not sure I agree with this defense in depth, because it would add\na lot to maintenance burden.\nSpecifically this part. The series that I sent out usually clashes\nhere as this is currently a hot area\nof the code touched by many different series in flight.\n\nHowever this is the long term correct thing to do? We assume no algorithm until\nthe repository can tell us from its config (or we default to sha1 if there is no\nconfiguration present).\n\nI wonder if there is yet another missing case in the enumeration of\nthe previous patch:\nSome commands are able to operate on GIT_OBJECT_DIR instead\nof GIT_DIR (git repack?), which may not even explore the full git directory,\nand so doesn't know about the hash value.\n\nIn the cover letter you reference my series, but comparing the diffstats\n(and looking through the patches), I would only expect this one place\nto have merge conflicts, which ought to be easy to resolve.\n(In my series I break the initializer into multiple lines to help the\nfuture, too)\n\nAfter some thought, I like this series.\n\nThanks,\nStefan\n"},{"id":"340061","messageId":"xmqqfu5rtrhg.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"CAGZ79karK7MT_qpFyYfVkioeBsBHidGdC_QfisPmG1a4dQuk5A@mail.gmail.com","subject":"Re: [PATCH 2/2] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-23T19:50:51Z","receivedAt":"2018-02-23T19:50:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> I wonder if there is yet another missing case in the enumeration of\n> the previous patch:\n> Some commands are able to operate on GIT_OBJECT_DIR instead\n> of GIT_DIR (git repack?), which may not even explore the full git directory,\n> and so doesn't know about the hash value.\n\n... because GIT_DIR/config is not known? \"repack\" is not one of\nthem, though---it needs to at least use refs as the starting point\nso a standalone OBJECT_DIR is insufficient.\n"},{"id":"340064","messageId":"CAGZ79kZjOz0RcBHZDrT3g3Rjkx=Q94aj81Y+PGmjcB11oAn16w@mail.gmail.com","threadId":"47823","inReplyTo":"xmqqy3jmxb8s.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 05/27] object-store: move packed_git and packed_git_mru to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-23T20:01:59Z","receivedAt":"2018-02-23T20:02:08Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Feb 21, 2018 at 1:51 PM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> +\n>> +/*\n>> + * The mru list_head is supposed to be initialized using\n>> + * the LIST_HEAD macro, assigning prev/next to itself.\n>> + * However this doesn't work in this case as some compilers dislike\n>> + * that macro on member variables. Use NULL instead as that is defined\n>> + * and accepted, deferring the real init to prepare_packed_git_mru(). */\n>> +#define __MRU_INIT { NULL, NULL }\n>> +#define RAW_OBJECT_STORE_INIT { NULL, NULL, __MRU_INIT, NULL, NULL }\n>\n> I do not think it has to be this way to abuse two NULLs, if you\n> faithfully mimicked how LIST_HEAD() macro is constructed. The\n> reason why it does not try to introduce\n>\n> struct list_head x = LIST_HEAD_INIT;\n>\n> and instead, uses\n>\n> LIST_HEAD(x);\n>\n> is because of the need for self referral. If we learn from it, we\n> can do the same, i.e. instead of doing\n>\n> struct raw_object_store x = RAW_OBJECT_STORE_INIT;\n>\n> we can do\n>\n> RAW_OBJECT_STORE(x);\n>\n> that expands to\n>\n> struct raw_object_store x = {\n> ...\n> { &x.packed_git_mru, &x.packed_git_mru },\n> ...\n> };\n>\n> no? Then we do not need such a lengthy comment that reads only like\n> an excuse ;-)\n\nWe cannot do this, because the object store is directly\nembedded into the the_repository (not as a pointer),\nwhich is a global on its own. So we'd have to do this\nat the repository level, which I would not want.\nI'd want to have the #defines where the structs are declared.\n\nAny guidance on how to do that correctly with self referential\ndefinitions without making the object store a pointer then?\n"},{"id":"340066","messageId":"CAGZ79kZUyo-C8FvphXh8dO8s0V5-Sa=fP=faXEiyD3xSezQsRw@mail.gmail.com","threadId":"47823","inReplyTo":"xmqqfu5rtrhg.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 2/2] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-23T20:04:22Z","receivedAt":"2018-02-23T20:04:29Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Feb 23, 2018 at 11:50 AM, Junio C Hamano wrote:\n> Stefan Beller writes:\n>\n>> I wonder if there is yet another missing case in the enumeration of\n>> the previous patch:\n>> Some commands are able to operate on GIT_OBJECT_DIR instead\n>> of GIT_DIR (git repack?), which may not even explore the full git directory,\n>> and so doesn't know about the hash value.\n>\n> ... because GIT_DIR/config is not known? \"repack\" is not one of\n> them, though---it needs to at least use refs as the starting point\n> so a standalone OBJECT_DIR is insufficient.\n\nYes, I could have worded this as a question:\nIs there any command that operates on GIT_OBJECT_DIR\nwithout trying to discover GIT_DIR ?\n"},{"id":"340079","messageId":"CAGZ79ka=6V-eicuQ=ZUrX7AE4iOWfwKJqeyLxp7bQs9z2sd95Q@mail.gmail.com","threadId":"47823","inReplyTo":"20180222064423.GD133592@aiede.svl.corp.google.com","subject":"Re: [PATCH 05/27] object-store: move packed_git and packed_git_mru to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-23T21:42:52Z","receivedAt":"2018-02-23T21:42:58Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":">> 2. Applying the semantic patch\n>> contrib/coccinelle/refactoring/packed_git.cocci to adjust callers.\n>> This semantic patch is placed in a sub directory of the coccinelle\n>> contrib dir, as this semantic patch is not expected to be of general\n>> usefulness; it is only useful during developing this series and\n>> merging it with other topics in flight. At a later date, just\n>> delete that semantic patch.\n>\n> Can the semantic patch go in the commit message instead? It is very\n> brief.\n\ndone\n\n>\n> Actually, I don't see this semantic patch in the diffstat. Is the\n> commit message stale?\n>\n>> 3. Applying line wrapping fixes from \"make style\" to break the\n>> resulting long lines.\n>>\n>> 4. Adding missing #includes of repository.h and object-store.h\n>> where needed.\n>\n> Is there a way to automate this step? (I'm asking for my own\n> reference when writing future patches, not because of any concern\n> about the correctness of this one.)\n\nno, for several reasons.\n(a) I don't know how to write it in coccinelle, because\n(b) I have not figured out the order in which we include headers, apart from\n \"cache.h goes first\", the rest of the includes sometimes looks like a random\n order, because different patches add new includes at different places.\n I have the impression, that (1) some add the include after all other\n existing includes or (2) try to figure out where to add the include to make\n sense in the existing include order or (3) sort alphabetically or (4) put it\n randomly, so the chance for a merge conflict with other series in flight\n decreases.\n(c) I did it in a semi automated fashion:\n while make fails:\n add another include\n\nThe mess with including repository or object-store comes from the fact that\nI had v2 based on object-store, not the repository and cherry-picked this patch\nover to v3. Fixed all of the includes now.\n\n>> @@ -59,10 +83,25 @@ struct raw_object_store {\n>> */\n>> char *objectdir;\n>>\n>> + struct packed_git *packed_git;\n>> + /*\n>> + * A most-recently-used ordered version of the packed_git list, which can\n>> + * be iterated instead of packed_git (and marked via mru_mark).\n>> + */\n>> + struct list_head packed_git_mru;\n>\n> I don't understand the new part of the comment. Can you explain here,\n> for me?\n\ncherrypicking error, fixed.\n\n>> -#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }\n>> +\n>> +/*\n>> + * The mru list_head is supposed to be initialized using\n>> + * the LIST_HEAD macro, assigning prev/next to itself.\n>> + * However this doesn't work in this case as some compilers dislike\n>> + * that macro on member variables. Use NULL instead as that is defined\n>> + * and accepted, deferring the real init to prepare_packed_git_mru(). */\n>\n> style nit: '*/' should be on its own line.\n>\n> More importantly, we can avoid such an issue as described by Junio. :)\n\nSee reply to Junio, I am not quite sure I like that.\n"},{"id":"340083","messageId":"20180223221746.GJ6619@genre.crustytoothpaste.net","threadId":"47823","inReplyTo":"20180223095640.25876-2-pclouds@gmail.com","subject":"Re: [PATCH 1/2] setup.c: initialize the_repository correctly in all cases","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2018-02-23T22:17:46Z","receivedAt":"2018-02-23T22:18:00Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Fri, Feb 23, 2018 at 04:56:39PM +0700, Nguyễn Thái Ngọc Duy wrote:\n> [3] the reason server side is still running ok with no hash algo\n> before [2] is because the programs that use enter_repo() do very\n> little then spawn a new program (like pack-objects or\n> upload-archive) to do the heavy lifting. These programs already\n> use setup_git_dir..()\n\nYou have \"..()\" here. Did you want to say \"().\" instead?\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\nhttps://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"340084","messageId":"CAGZ79kahS_mk8gFONUAxBjw7iV=5q1ff-ZgjLnQ7rMw=kBKThw@mail.gmail.com","threadId":"47823","inReplyTo":"20180222004710.GF127348@google.com","subject":"Re: [PATCH 08/27] pack: move approximate object count to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-23T22:22:14Z","receivedAt":"2018-02-23T22:22:21Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Feb 21, 2018 at 4:47 PM, Brandon Williams wrote:\n> On 02/20, Stefan Beller wrote:\n>> The approximate_object_count() function maintains a rough count of\n>> objects in a repository to estimate how long object name abbreviates\n>> should be. Object names are scoped to a repository and the\n>> appropriate length may differ by repository, so the object count\n>> should not be global.\n>>\n>> Signed-off-by: Stefan Beller \n>> Signed-off-by: Jonathan Nieder \n>> ---\n>> object-store.h | 10 +++++++++-\n>> packfile.c | 11 +++++------\n>> 2 files changed, 14 insertions(+), 7 deletions(-)\n>>\n>> diff --git a/object-store.h b/object-store.h\n>> index 6cecba3951..bd1e4fcd8b 100644\n>> --- a/object-store.h\n>> +++ b/object-store.h\n>> @@ -93,6 +93,14 @@ struct raw_object_store {\n>> struct alternate_object_database *alt_odb_list;\n>> struct alternate_object_database **alt_odb_tail;\n>>\n>> + /*\n>> + * A fast, rough count of the number of objects in the repository.\n>> + * These two fields are not meant for direct access. Use\n>> + * approximate_object_count() instead.\n>> + */\n>> + unsigned long approximate_object_count;\n>> + unsigned approximate_object_count_valid : 1;\n>\n> Patch looks fine and is effectively a no-op, though what is the need for\n> both of these variables? Maybe it can be simplified down to just use\n> one? Just musing as its out of the scope of this patch and we probably\n> shouldn't try to change that in this series.\n\nI agree we should. It was introduced in e323de4ad7f (sha1_file:\nallow sha1_loose_object_info to handle arbitrary repositories, 2017-08-30)\nand I think it was seen as a clever optimization trick back then?\n"},{"id":"340085","messageId":"xmqqfu5rs5qb.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"CAGZ79kZUyo-C8FvphXh8dO8s0V5-Sa=fP=faXEiyD3xSezQsRw@mail.gmail.com","subject":"Re: [PATCH 2/2] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-23T22:26:04Z","receivedAt":"2018-02-23T22:26:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> On Fri, Feb 23, 2018 at 11:50 AM, Junio C Hamano wrote:\n>> Stefan Beller writes:\n>>\n>>> I wonder if there is yet another missing case in the enumeration of\n>>> the previous patch:\n>>> Some commands are able to operate on GIT_OBJECT_DIR instead\n>>> of GIT_DIR (git repack?), which may not even explore the full git directory,\n>>> and so doesn't know about the hash value.\n>>\n>> ... because GIT_DIR/config is not known? \"repack\" is not one of\n>> them, though---it needs to at least use refs as the starting point\n>> so a standalone OBJECT_DIR is insufficient.\n>\n> Yes, I could have worded this as a question:\n> Is there any command that operates on GIT_OBJECT_DIR\n> without trying to discover GIT_DIR ?\n\nIf somebody finds one that would be a good argument not to pursue\nthe approach. Lack of response to the question would not amount to\nthat much---it is possible all people who bothered to think of\noverlooked something obvious, though. \"When I asked around nobody\nthought of a possibly way for this to cause breakages, so let's\ndeclare it is safe to do so and do it\" is not how we want to do\nthings X-<.\n\n"},{"id":"340087","messageId":"20180223222950.GK6619@genre.crustytoothpaste.net","threadId":"47823","inReplyTo":"20180223095640.25876-3-pclouds@gmail.com","subject":"Re: [PATCH 2/2] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2018-02-23T22:29:50Z","receivedAt":"2018-02-23T22:30:07Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Fri, Feb 23, 2018 at 04:56:40PM +0700, Nguyễn Thái Ngọc Duy wrote:\n> diff --git a/builtin/index-pack.c b/builtin/index-pack.c\n> index 7e3e1a461c..8ee935504e 100644\n> --- a/builtin/index-pack.c\n> +++ b/builtin/index-pack.c\n> @@ -1673,6 +1673,11 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n> \tif (prefix && chdir(prefix))\n> \t\tdie(_(\"Cannot come back to cwd\"));\n> \n> +\tif (!the_hash_algo) {\n> +\t\twarning(_(\"Running without a repository, assuming SHA-1 hash\"));\n> +\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n> +\t}\n\nIs this warning going to be visible to users in the normal course of\noperation? If so, people are probably going to find this bothersome or\nalarming.\n\n> \tfor (i = 1; i < argc; i++) {\n> \t\tconst char *arg = argv[i];\n> \n> diff --git a/common-main.c b/common-main.c\n> index 6a689007e7..12aec36794 100644\n> --- a/common-main.c\n> +++ b/common-main.c\n> @@ -1,6 +1,7 @@\n> #include \"cache.h\"\n> #include \"exec_cmd.h\"\n> #include \"attr.h\"\n> +#include \"repository.h\"\n> \n> /*\n> * Many parts of Git have subprograms communicate via pipe, expect the\n> @@ -40,5 +41,8 @@ int main(int argc, const char **argv)\n> \n> \trestore_sigpipe_to_default();\n> \n> +\tif (getenv(\"GIT_HASH_FIXUP\"))\n> +\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n\nI'm lukewarm on adding this environment variable, but considering our\nhistory here, we had probably better. We can always remove it after a\nfew releases.\n\n> \treturn cmd_main(argc, argv);\n> }\n> diff --git a/diff-no-index.c b/diff-no-index.c\n> index 0ed5f0f496..f038f665bc 100644\n> --- a/diff-no-index.c\n> +++ b/diff-no-index.c\n> @@ -241,6 +241,11 @@ void diff_no_index(struct rev_info *revs,\n> \tstruct strbuf replacement = STRBUF_INIT;\n> \tconst char *prefix = revs->prefix;\n> \n> +\tif (!the_hash_algo) {\n> +\t\twarning(_(\"Running without a repository, assuming SHA-1 hash\"));\n> +\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n> +\t}\n\nAgain, same concern. I can imagine scripts that will blow up loudly if\ngit diff --no-index spews things to standard error.\n\nI'm not opposed to making this more visible, but I wonder if maybe it\nshould only be visible to developers or such. The only way I can think\nof doing is that is with an advice options, but maybe there's a better\nway.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\nhttps://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"340088","messageId":"CAGZ79kYozWRw5rtCSN+Fab2x+MDSOSASdY4MZgSykw9cN98qaw@mail.gmail.com","threadId":"47823","inReplyTo":"20180222065122.GE133592@aiede.svl.corp.google.com","subject":"Re: [PATCH 13/27] sha1_file: add repository argument to prepare_alt_odb","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-23T22:33:27Z","receivedAt":"2018-02-23T22:33:34Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":">> --- a/sha1_file.c\n>> +++ b/sha1_file.c\n>> @@ -23,6 +23,7 @@\n>> #include \"sha1-lookup.h\"\n>> #include \"bulk-checkin.h\"\n>> #include \"repository.h\"\n>> +#include \"object-store.h\"\n>\n> Should this #include have been added in an earlier patch, since the\n> file both uses and defines prepare_alt_odb, which is declared there?\n\nThis is another superflous include, dropped it. Thanks for catching.\n"},{"id":"340089","messageId":"CAGZ79kZess5QJL1RuV+qmUYtLoY0q0EJkvRF1rZPWeScFcRz_w@mail.gmail.com","threadId":"47823","inReplyTo":"20180222005149.GH127348@google.com","subject":"Re: [PATCH 16/27] sha1_file: add repository argument to sha1_file_name","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-23T22:36:38Z","receivedAt":"2018-02-23T22:36:43Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Feb 21, 2018 at 4:51 PM, Brandon Williams wrote:\n> On 02/20, Stefan Beller wrote:\n>> Add a repository argument to allow sha1_file_name callers to be more\n>> specific about which repository to handle. This is a small mechanical\n>> change; it doesn't change the implementation to handle repositories\n>> other than the_repository yet.\n>>\n>> As with the previous commits, use a macro to catch callers passing a\n>> repository other than the_repository at compile time.\n>>\n>> While at it, move the declaration to object-store.h, where it should\n>> be easier to find.\n>\n> Seems like we may want to make a sha1-file.h or an oid-file.h or\n> something like that at some point as that seems like a better place for\n> the function than in the object-store.h file?\n\nIt depends what our long term goal is.\nDo we want header and source file name to match for each function?\nOr do we want a coarser set of headers, such that we have a broad\nobject-store.h, but that is implemented in sha1_file.c, packfile.c\nfor the parts of the raw_objectstore and other .c files for the higher\nlevels of the object store?\n\nFor now I'd just keep it in object-store.h as moving out just a couple\nfunctions seems less consistent?\n\nStefan\n"},{"id":"340091","messageId":"20180223224749.GL6619@genre.crustytoothpaste.net","threadId":"47823","inReplyTo":"20180223095640.25876-1-pclouds@gmail.com","subject":"Re: [PATCH 0/2] Fix initializing the_hash_algo","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2018-02-23T22:47:49Z","receivedAt":"2018-02-23T22:48:02Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Fri, Feb 23, 2018 at 04:56:38PM +0700, Nguyễn Thái Ngọc Duy wrote:\n> I can certainly try! I start to remember all the hairy details in that\n> setup code.\n> \n> The first step may be something like this, which identifies all the\n> \"repo init\" entry points. This is basically a revert of e26f7f19b6\n> (repository: pre-initialize hash algo pointer - 2018-01-19) and doing\n> things the proper way, hopefully.\n> \n> This is on 'master', independent from Stefan's series. I have another\n> patch on top of that series to remove the use of ignore_env in\n> sha1_file.c (and things seem to work). Basically whenever you have to\n> initialize the hash algorithm, there's a good chance you need to\n> initialize object store as well. But I'll hold that off until\n> Stefan's and this one are both merged.\n\nI definitely think this series is an improvement over my previous patch.\n\nMy major concern is alarming users (or breaking scripts) with the\nwarning message. I wonder if deferring the use of the warning until we\nhave multiple hash algorithms might be a better idea. At that point,\nthe warning would become something people could act upon.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\nhttps://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"340092","messageId":"CAGZ79kaSTz+CjpZ8cijRcK_npWnh1TK9btzgWAXPHUDYKx--cQ@mail.gmail.com","threadId":"47823","inReplyTo":"20180221164406.c98170765e454b51de80c680@google.com","subject":"Re: [PATCH 22/27] sha1_file: allow sha1_file_name to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-23T22:50:19Z","receivedAt":"2018-02-23T22:50:25Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Feb 21, 2018 at 4:44 PM, Jonathan Tan wrote:\n> On Tue, 20 Feb 2018 17:54:25 -0800\n> Stefan Beller wrote:\n>\n>> Signed-off-by: Stefan Beller \n>> Signed-off-by: Jonathan Nieder \n>\n> Reviewed-by: Jonathan Tan \n>\n>> -void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)\n>> +void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1)\n>> {\n>> - strbuf_addstr(buf, get_object_directory());\n>> + strbuf_addstr(buf, r->objects.objectdir);\n>> strbuf_addch(buf, '/');\n>> fill_sha1_path(buf, sha1);\n>> }\n>\n> In the future, we should probably have:\n> - a function to get the object store out of a repo (so that it can\n> lazily initialize the object store struct if necessary)\n> - when the object store is obtained, its objectdir field is guaranteed\n> to be populated\n> - sha1_file_name should take the object store struct, not the repo\n> struct\n\nWe had that in v2, but I reverted it as the consensus seemed that we prefer\nthe_repository to be passed around.\n\n>\n> but this is outside the scope of this patch.\n"},{"id":"340095","messageId":"20180223231142.GH234838@google.com","threadId":"47823","inReplyTo":"CAGZ79kZess5QJL1RuV+qmUYtLoY0q0EJkvRF1rZPWeScFcRz_w@mail.gmail.com","subject":"Re: [PATCH 16/27] sha1_file: add repository argument to sha1_file_name","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-23T23:11:42Z","receivedAt":"2018-02-23T23:11:50Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/23, Stefan Beller wrote:\n> On Wed, Feb 21, 2018 at 4:51 PM, Brandon Williams wrote:\n> > On 02/20, Stefan Beller wrote:\n> >> Add a repository argument to allow sha1_file_name callers to be more\n> >> specific about which repository to handle. This is a small mechanical\n> >> change; it doesn't change the implementation to handle repositories\n> >> other than the_repository yet.\n> >>\n> >> As with the previous commits, use a macro to catch callers passing a\n> >> repository other than the_repository at compile time.\n> >>\n> >> While at it, move the declaration to object-store.h, where it should\n> >> be easier to find.\n> >\n> > Seems like we may want to make a sha1-file.h or an oid-file.h or\n> > something like that at some point as that seems like a better place for\n> > the function than in the object-store.h file?\n> \n> It depends what our long term goal is.\n> Do we want header and source file name to match for each function?\n> Or do we want a coarser set of headers, such that we have a broad\n> object-store.h, but that is implemented in sha1_file.c, packfile.c\n> for the parts of the raw_objectstore and other .c files for the higher\n> levels of the object store?\n> \n> For now I'd just keep it in object-store.h as moving out just a couple\n> functions seems less consistent?\n\nFair enough :)\n\n-- \nBrandon Williams\n"},{"id":"340096","messageId":"20180223231608.GI234838@google.com","threadId":"47823","inReplyTo":"20180223222950.GK6619@genre.crustytoothpaste.net","subject":"Re: [PATCH 2/2] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-23T23:16:08Z","receivedAt":"2018-02-23T23:16:16Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/23, brian m. carlson wrote:\n> On Fri, Feb 23, 2018 at 04:56:40PM +0700, Nguyễn Thái Ngọc Duy wrote:\n> > diff --git a/builtin/index-pack.c b/builtin/index-pack.c\n> > index 7e3e1a461c..8ee935504e 100644\n> > --- a/builtin/index-pack.c\n> > +++ b/builtin/index-pack.c\n> > @@ -1673,6 +1673,11 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)\n> > \tif (prefix && chdir(prefix))\n> > \t\tdie(_(\"Cannot come back to cwd\"));\n> > \n> > +\tif (!the_hash_algo) {\n> > +\t\twarning(_(\"Running without a repository, assuming SHA-1 hash\"));\n> > +\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n> > +\t}\n> \n> Is this warning going to be visible to users in the normal course of\n> operation? If so, people are probably going to find this bothersome or\n> alarming.\n> \n> > \tfor (i = 1; i < argc; i++) {\n> > \t\tconst char *arg = argv[i];\n> > \n> > diff --git a/common-main.c b/common-main.c\n> > index 6a689007e7..12aec36794 100644\n> > --- a/common-main.c\n> > +++ b/common-main.c\n> > @@ -1,6 +1,7 @@\n> > #include \"cache.h\"\n> > #include \"exec_cmd.h\"\n> > #include \"attr.h\"\n> > +#include \"repository.h\"\n> > \n> > /*\n> > * Many parts of Git have subprograms communicate via pipe, expect the\n> > @@ -40,5 +41,8 @@ int main(int argc, const char **argv)\n> > \n> > \trestore_sigpipe_to_default();\n> > \n> > +\tif (getenv(\"GIT_HASH_FIXUP\"))\n> > +\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n> \n> I'm lukewarm on adding this environment variable, but considering our\n> history here, we had probably better. We can always remove it after a\n> few releases.\n\nYeah I don't know what the best thing to do here would be. I mean we\ncould always just init the hash_algo for the_repository here so that we\ndon't ever have to worry about it, but I don't know if even that is the\nright approach.\n\n> \n> > \treturn cmd_main(argc, argv);\n> > }\n> > diff --git a/diff-no-index.c b/diff-no-index.c\n> > index 0ed5f0f496..f038f665bc 100644\n> > --- a/diff-no-index.c\n> > +++ b/diff-no-index.c\n> > @@ -241,6 +241,11 @@ void diff_no_index(struct rev_info *revs,\n> > \tstruct strbuf replacement = STRBUF_INIT;\n> > \tconst char *prefix = revs->prefix;\n> > \n> > +\tif (!the_hash_algo) {\n> > +\t\twarning(_(\"Running without a repository, assuming SHA-1 hash\"));\n> > +\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n> > +\t}\n> \n> Again, same concern. I can imagine scripts that will blow up loudly if\n> git diff --no-index spews things to standard error.\n> \n> I'm not opposed to making this more visible, but I wonder if maybe it\n> should only be visible to developers or such. The only way I can think\n> of doing is that is with an advice options, but maybe there's a better\n> way.\n> -- \n> brian m. carlson / brian with sandals: Houston, Texas, US\n> https://www.crustytoothpaste.net/~bmc | My opinion only\n> OpenPGP: https://keybase.io/bk2204\n\n\n\n-- \nBrandon Williams\n"},{"id":"340097","messageId":"CAGZ79kZBP3tVEAFwDMGcSnuHYk2TUnAbwf3qowzJ-=hb8PdE3w@mail.gmail.com","threadId":"47823","inReplyTo":"20180221163530.3a6659b8e0bec5dfab835232@google.com","subject":"Re: [PATCH 15/27] sha1_file: allow prepare_alt_odb to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-23T23:18:25Z","receivedAt":"2018-02-23T23:18:30Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Feb 21, 2018 at 4:35 PM, Jonathan Tan wrote:\n> On Tue, 20 Feb 2018 17:54:18 -0800\n> Stefan Beller wrote:\n>\n>> -void prepare_alt_odb_the_repository(void)\n>> +void prepare_alt_odb(struct repository *r)\n>> {\n>> - const char *alt;\n>> -\n>> - if (the_repository->objects.alt_odb_tail)\n>> + if (r->objects.alt_odb_tail)\n>> return;\n>>\n>> - alt = getenv(ALTERNATE_DB_ENVIRONMENT);\n>> + r->objects.alt_odb_tail = &r->objects.alt_odb_list;\n>> +\n>> + if (!r->ignore_env) {\n>> + const char *alt = getenv(ALTERNATE_DB_ENVIRONMENT);\n>> + if (!alt)\n>> + alt = \"\";\n>\n> alt can be NULL, just like in the existing code, so these 2 lines are\n> unnecessary. (link_alt_odb_entries() will just do nothing in either\n> case.)\n>\n> (I also think that the check of absence of alt should be done by the\n> caller of link_alt_odb_entries(), not by link_alt_odb_entries() itself,\n> but that is much beyond the scope of this patch set.)\n>\n\nreverted to a literal translation of s/the_repository/r/, I forget\nwhy I implemented this change in the first place. dropped for now.\n\nThanks,\nStefan\n"},{"id":"340115","messageId":"20180224004754.129721-1-sbeller@google.com","threadId":"47823","inReplyTo":"20180221015430.96054-1-sbeller@google.com","subject":"[PATCHv4 00/27] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:27Z","receivedAt":"2018-02-24T00:48:04Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"v4:\n* addressed feedback from the single patches (mostly nits)\n* rebased on latest master\n\nv3:\n* reverted back to use the repository for most of the functions\n (including unduplicating 'ignore_env')\n* rebased on master again (I lost that state when doing v2, as\n I did both rebase as well as conversion to object store in one go)\n* one additional patch, that moves the alternates related things to\n object-store.h, such that inclusion of cache.h (in object-store.h)\n is not required any more.\n\nv2:\n* duplicated the 'ignore_env' bit into the object store as well\n* the #define trick no longer works as we do not have a \"the_objectstore\" global,\n which means there is just one patch per function that is converted.\n As this follows the same structure of the previous series, I am still confident\n there is no hidden dependencies to globals outside the object store in these\n converted functions.\n* rebased on top of current master, resolving the merge conflicts.\n I think I used the list.h APIs right, but please double check.\n\nThanks,\nStefan\n\nv1:\nThis is a real take on the first part of the recent RFC[1].\n\nJonathan Tan suggested[2] that \"sha1_loose_object_info to handle arbitrary repositories\"\nmight be a good breaking point for a first part at that RFC at patch 38.\nThis series is smaller and contains only 26 patches as the patches in the big\nRFC were slightly out of order.\n\nI developed this series partly by writing patches, but mostly by cherrypicking\nfrom that RFC on top of current master. I noticed no external conflicts apart\nfrom one addition to the repositories _INIT macro, which was easy to resolve.\n\nComments in the early range of that RFC were on 003 where Junio pointed out\nthat the coccinelle patch ought to be not in contrib/coccinelle, so I put it\nin a sub directory there, as 'make coccicheck' doesn't traverse subdirs.\n\nbrian had a questoin on patch 25 in the RFC, but that seemed to resolve itself\nwithout any suggestion to include into this series[3].\n\nDuy suggested that we shall not use the repository blindly, but should carefully\nexamine whether to pass on an object store or the refstore or such[4], which\nI agree with if it makes sense. This series unfortunately has an issue with that\nas I would not want to pass down the `ignore_env` flag separately from the object\nstore, so I made all functions that only take the object store to have the raw\nobject store as the first parameter, and others using the full repository.\n\nEric Sunshine brought up memory leaks with the RFC, and I would think to\nhave plugged all holes.\n\n[1] https://public-inbox.org/git/20180205235508.216277-1-sbeller@google.com/\n[2] https://public-inbox.org/git/20180207143300.ce1c39ca07f6a0d64fe0e7ca@google.com/\n[3] https://public-inbox.org/git/20180206011940.GD7904@genre.crustytoothpaste.net/\n[4] https://public-inbox.org/git/CACsJy8CGgekpX4cZkyyTSPrj87uQVKZSOL7fyT__P2dh_1LmVQ@mail.gmail.com/\n\nThanks,\nStefan\n\nJonathan Nieder (2):\n sha1_file: allow map_sha1_file_1 to handle arbitrary repositories\n sha1_file: allow sha1_loose_object_info to handle arbitrary\n repositories\n\nStefan Beller (25):\n repository: introduce raw object store field\n object-store: migrate alternates struct and functions from cache.h\n object-store: move alt_odb_list and alt_odb_tail to object store\n object-store: free alt_odb_list\n object-store: move packed_git and packed_git_mru to object store\n object-store: close all packs upon clearing the object store\n pack: move prepare_packed_git_run_once to object store\n pack: move approximate object count to object store\n sha1_file: add raw_object_store argument to alt_odb_usable\n sha1_file: add repository argument to link_alt_odb_entry\n sha1_file: add repository argument to read_info_alternates\n sha1_file: add repository argument to link_alt_odb_entries\n sha1_file: add repository argument to prepare_alt_odb\n sha1_file: allow link_alt_odb_entries to handle arbitrary repositories\n sha1_file: allow prepare_alt_odb to handle arbitrary repositories\n sha1_file: add repository argument to sha1_file_name\n sha1_file: add repository argument to stat_sha1_file\n sha1_file: add repository argument to open_sha1_file\n sha1_file: add repository argument to map_sha1_file_1\n sha1_file: add repository argument to map_sha1_file\n sha1_file: add repository argument to sha1_loose_object_info\n sha1_file: allow sha1_file_name to handle arbitrary repositories\n sha1_file: allow stat_sha1_file to handle arbitrary repositories\n sha1_file: allow open_sha1_file to handle arbitrary repositories\n sha1_file: allow map_sha1_file to handle arbitrary repositories\n\n builtin/am.c | 2 +-\n builtin/clone.c | 2 +-\n builtin/count-objects.c | 5 +-\n builtin/fetch.c | 2 +-\n builtin/fsck.c | 12 ++--\n builtin/gc.c | 3 +-\n builtin/grep.c | 2 +-\n builtin/merge.c | 2 +-\n builtin/pack-objects.c | 20 ++++---\n builtin/pack-redundant.c | 5 +-\n builtin/receive-pack.c | 2 +-\n cache.h | 87 ---------------------------\n environment.c | 5 +-\n fast-import.c | 7 ++-\n http-backend.c | 5 +-\n http-walker.c | 3 +-\n http.c | 5 +-\n object-store.h | 124 ++++++++++++++++++++++++++++++++++++++\n object.c | 27 +++++++++\n pack-bitmap.c | 3 +-\n packfile.c | 64 ++++++++++----------\n packfile.h | 2 +-\n path.c | 2 +-\n repository.c | 21 +++++--\n repository.h | 7 ++-\n server-info.c | 5 +-\n sha1_file.c | 125 +++++++++++++++++++++------------------\n sha1_name.c | 10 ++--\n streaming.c | 5 +-\n 29 files changed, 335 insertions(+), 229 deletions(-)\n create mode 100644 object-store.h\n\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340116","messageId":"20180224004754.129721-2-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 01/27] repository: introduce raw object store field","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:28Z","receivedAt":"2018-02-24T00:48:07Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The raw object store field will contain any objects needed for\naccess to objects in a given repository.\n\nThis patch introduces the raw object store and populates it with the\n`objectdir`, which used to be part of the repository struct.\n\nAs the struct gains members, we'll also populate the function to clear\nthe memory for these members.\n\nIn a later we'll introduce a struct object_parser, that will complement\nthe object parsing in a repository struct: The raw object parser is the\nlayer that will provide access to raw object content, while the higher\nlevel object parser code will parse raw objects and keeps track of\nparenthood and other object relationships using 'struct object'.\nFor now only add the lower level to the repository struct.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/grep.c | 2 +-\n environment.c | 5 +++--\n object-store.h | 15 +++++++++++++++\n object.c | 5 +++++\n path.c | 2 +-\n repository.c | 19 ++++++++++++++-----\n repository.h | 7 ++++---\n 7 files changed, 43 insertions(+), 12 deletions(-)\n create mode 100644 object-store.h\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 3ca4ac80d8c..0f0c195705f 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -432,7 +432,7 @@ static int grep_submodule(struct grep_opt *opt, struct repository *superproject,\n \t * object.\n \t */\n \tgrep_read_lock();\n-\tadd_to_alternates_memory(submodule.objectdir);\n+\tadd_to_alternates_memory(submodule.objects.objectdir);\n \tgrep_read_unlock();\n \n \tif (oid) {\ndiff --git a/environment.c b/environment.c\nindex de8431e01e7..ec10b062e68 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -13,6 +13,7 @@\n #include \"refs.h\"\n #include \"fmt-merge-msg.h\"\n #include \"commit.h\"\n+#include \"object-store.h\"\n \n int trust_executable_bit = 1;\n int trust_ctime = 1;\n@@ -244,9 +245,9 @@ const char *get_git_work_tree(void)\n \n char *get_object_directory(void)\n {\n-\tif (!the_repository->objectdir)\n+\tif (!the_repository->objects.objectdir)\n \t\tBUG(\"git environment hasn't been setup\");\n-\treturn the_repository->objectdir;\n+\treturn the_repository->objects.objectdir;\n }\n \n int odb_mkstemp(struct strbuf *template, const char *pattern)\ndiff --git a/object-store.h b/object-store.h\nnew file mode 100644\nindex 00000000000..cf35760ceb6\n--- /dev/null\n+++ b/object-store.h\n@@ -0,0 +1,15 @@\n+#ifndef OBJECT_STORE_H\n+#define OBJECT_STORE_H\n+\n+struct raw_object_store {\n+\t/*\n+\t * Path to the repository's object store.\n+\t * Cannot be NULL after initialization.\n+\t */\n+\tchar *objectdir;\n+};\n+#define RAW_OBJECT_STORE_INIT { NULL }\n+\n+void raw_object_store_clear(struct raw_object_store *o);\n+\n+#endif /* OBJECT_STORE_H */\ndiff --git a/object.c b/object.c\nindex 9e6f9ff20b0..a2acdee1405 100644\n--- a/object.c\n+++ b/object.c\n@@ -445,3 +445,8 @@ void clear_commit_marks_all(unsigned int flags)\n \t\t\tobj->flags &= ~flags;\n \t}\n }\n+\n+void raw_object_store_clear(struct raw_object_store *o)\n+{\n+\tFREE_AND_NULL(o->objectdir);\n+}\ndiff --git a/path.c b/path.c\nindex da8b655730d..81a42d91155 100644\n--- a/path.c\n+++ b/path.c\n@@ -382,7 +382,7 @@ static void adjust_git_path(const struct repository *repo,\n \t\tstrbuf_splice(buf, 0, buf->len,\n \t\t\t repo->index_file, strlen(repo->index_file));\n \telse if (dir_prefix(base, \"objects\"))\n-\t\treplace_dir(buf, git_dir_len + 7, repo->objectdir);\n+\t\treplace_dir(buf, git_dir_len + 7, repo->objects.objectdir);\n \telse if (git_hooks_path && dir_prefix(base, \"hooks\"))\n \t\treplace_dir(buf, git_dir_len + 5, git_hooks_path);\n \telse if (repo->different_commondir)\ndiff --git a/repository.c b/repository.c\nindex 4ffbe9bc94e..fb65e8072d5 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -1,11 +1,18 @@\n #include \"cache.h\"\n #include \"repository.h\"\n+#include \"object-store.h\"\n #include \"config.h\"\n #include \"submodule-config.h\"\n \n /* The main repository */\n static struct repository the_repo = {\n-\tNULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0\n+\tNULL, NULL,\n+\tRAW_OBJECT_STORE_INIT,\n+\tNULL, NULL, NULL,\n+\tNULL, NULL, NULL,\n+\t&the_index,\n+\t&hash_algos[GIT_HASH_SHA1],\n+\t0, 0\n };\n struct repository *the_repository = &the_repo;\n \n@@ -42,9 +49,10 @@ static void repo_setup_env(struct repository *repo)\n \t\t\t\t\t\t !repo->ignore_env);\n \tfree(repo->commondir);\n \trepo->commondir = strbuf_detach(&sb, NULL);\n-\tfree(repo->objectdir);\n-\trepo->objectdir = git_path_from_env(DB_ENVIRONMENT, repo->commondir,\n-\t\t\t\t\t \"objects\", !repo->ignore_env);\n+\traw_object_store_clear(&repo->objects);\n+\trepo->objects.objectdir =\n+\t\tgit_path_from_env(DB_ENVIRONMENT, repo->commondir,\n+\t\t\t\t \"objects\", !repo->ignore_env);\n \tfree(repo->graft_file);\n \trepo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,\n \t\t\t\t\t \"info/grafts\", !repo->ignore_env);\n@@ -209,12 +217,13 @@ void repo_clear(struct repository *repo)\n {\n \tFREE_AND_NULL(repo->gitdir);\n \tFREE_AND_NULL(repo->commondir);\n-\tFREE_AND_NULL(repo->objectdir);\n \tFREE_AND_NULL(repo->graft_file);\n \tFREE_AND_NULL(repo->index_file);\n \tFREE_AND_NULL(repo->worktree);\n \tFREE_AND_NULL(repo->submodule_prefix);\n \n+\traw_object_store_clear(&repo->objects);\n+\n \tif (repo->config) {\n \t\tgit_configset_clear(repo->config);\n \t\tFREE_AND_NULL(repo->config);\ndiff --git a/repository.h b/repository.h\nindex 0329e40c7f5..fa73ab8e938 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -1,6 +1,8 @@\n #ifndef REPOSITORY_H\n #define REPOSITORY_H\n \n+#include \"object-store.h\"\n+\n struct config_set;\n struct index_state;\n struct submodule_cache;\n@@ -21,10 +23,9 @@ struct repository {\n \tchar *commondir;\n \n \t/*\n-\t * Path to the repository's object store.\n-\t * Cannot be NULL after initialization.\n+\t * Holds any information related to accessing the raw object content.\n \t */\n-\tchar *objectdir;\n+\tstruct raw_object_store objects;\n \n \t/*\n \t * Path to the repository's graft file.\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340117","messageId":"20180224004754.129721-3-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 02/27] object-store: migrate alternates struct and functions from cache.h","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:29Z","receivedAt":"2018-02-24T00:48:10Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Migrate the struct alternate_object_database and all its related\nfunctions to the object store as these functions are easier found in\nthat header. The migration is just a verbatim copy, no need to\ninclude the object store header at any C file, because cache.h includes\nrepository.h which in turn includes the object-store.h\n\nSigned-off-by: Stefan Beller \n---\n cache.h | 51 --------------------------------------------------\n object-store.h | 51 ++++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 51 insertions(+), 51 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 21fbcc24149..99eb2bdb893 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1560,57 +1560,6 @@ extern int has_dirs_only_path(const char *name, int len, int prefix_len);\n extern void schedule_dir_for_removal(const char *name, int len);\n extern void remove_scheduled_dirs(void);\n \n-extern struct alternate_object_database {\n-\tstruct alternate_object_database *next;\n-\n-\t/* see alt_scratch_buf() */\n-\tstruct strbuf scratch;\n-\tsize_t base_len;\n-\n-\t/*\n-\t * Used to store the results of readdir(3) calls when searching\n-\t * for unique abbreviated hashes. This cache is never\n-\t * invalidated, thus it's racy and not necessarily accurate.\n-\t * That's fine for its purpose; don't use it for tasks requiring\n-\t * greater accuracy!\n-\t */\n-\tchar loose_objects_subdir_seen[256];\n-\tstruct oid_array loose_objects_cache;\n-\n-\tchar path[FLEX_ARRAY];\n-} *alt_odb_list;\n-extern void prepare_alt_odb(void);\n-extern char *compute_alternate_path(const char *path, struct strbuf *err);\n-typedef int alt_odb_fn(struct alternate_object_database *, void *);\n-extern int foreach_alt_odb(alt_odb_fn, void*);\n-\n-/*\n- * Allocate a \"struct alternate_object_database\" but do _not_ actually\n- * add it to the list of alternates.\n- */\n-struct alternate_object_database *alloc_alt_odb(const char *dir);\n-\n-/*\n- * Add the directory to the on-disk alternates file; the new entry will also\n- * take effect in the current process.\n- */\n-extern void add_to_alternates_file(const char *dir);\n-\n-/*\n- * Add the directory to the in-memory list of alternates (along with any\n- * recursive alternates it points to), but do not modify the on-disk alternates\n- * file.\n- */\n-extern void add_to_alternates_memory(const char *dir);\n-\n-/*\n- * Returns a scratch strbuf pre-filled with the alternate object directory,\n- * including a trailing slash, which can be used to access paths in the\n- * alternate. Always use this over direct access to alt->scratch, as it\n- * cleans up any previous use of the scratch buffer.\n- */\n-extern struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);\n-\n struct pack_window {\n \tstruct pack_window *next;\n \tunsigned char *base;\ndiff --git a/object-store.h b/object-store.h\nindex cf35760ceb6..5678aa1136c 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -1,6 +1,57 @@\n #ifndef OBJECT_STORE_H\n #define OBJECT_STORE_H\n \n+extern struct alternate_object_database {\n+\tstruct alternate_object_database *next;\n+\n+\t/* see alt_scratch_buf() */\n+\tstruct strbuf scratch;\n+\tsize_t base_len;\n+\n+\t/*\n+\t * Used to store the results of readdir(3) calls when searching\n+\t * for unique abbreviated hashes. This cache is never\n+\t * invalidated, thus it's racy and not necessarily accurate.\n+\t * That's fine for its purpose; don't use it for tasks requiring\n+\t * greater accuracy!\n+\t */\n+\tchar loose_objects_subdir_seen[256];\n+\tstruct oid_array loose_objects_cache;\n+\n+\tchar path[FLEX_ARRAY];\n+} *alt_odb_list;\n+void prepare_alt_odb(void);\n+char *compute_alternate_path(const char *path, struct strbuf *err);\n+typedef int alt_odb_fn(struct alternate_object_database *, void *);\n+int foreach_alt_odb(alt_odb_fn, void*);\n+\n+/*\n+ * Allocate a \"struct alternate_object_database\" but do _not_ actually\n+ * add it to the list of alternates.\n+ */\n+struct alternate_object_database *alloc_alt_odb(const char *dir);\n+\n+/*\n+ * Add the directory to the on-disk alternates file; the new entry will also\n+ * take effect in the current process.\n+ */\n+void add_to_alternates_file(const char *dir);\n+\n+/*\n+ * Add the directory to the in-memory list of alternates (along with any\n+ * recursive alternates it points to), but do not modify the on-disk alternates\n+ * file.\n+ */\n+void add_to_alternates_memory(const char *dir);\n+\n+/*\n+ * Returns a scratch strbuf pre-filled with the alternate object directory,\n+ * including a trailing slash, which can be used to access paths in the\n+ * alternate. Always use this over direct access to alt->scratch, as it\n+ * cleans up any previous use of the scratch buffer.\n+ */\n+struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);\n+\n struct raw_object_store {\n \t/*\n \t * Path to the repository's object store.\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340118","messageId":"20180224004754.129721-5-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 04/27] object-store: free alt_odb_list","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:31Z","receivedAt":"2018-02-24T00:48:11Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Free the memory and reset alt_odb_{list, tail} to NULL.\n\nSigned-off-by: Stefan Beller \n---\n object.c | 17 +++++++++++++++++\n 1 file changed, 17 insertions(+)\n\ndiff --git a/object.c b/object.c\nindex a2acdee1405..920dc4463fa 100644\n--- a/object.c\n+++ b/object.c\n@@ -446,7 +446,24 @@ void clear_commit_marks_all(unsigned int flags)\n \t}\n }\n \n+static void free_alt_odb(struct alternate_object_database *alt)\n+{\n+\tstrbuf_release(&alt->scratch);\n+\toid_array_clear(&alt->loose_objects_cache);\n+}\n+\n+static void free_alt_odbs(struct raw_object_store *o)\n+{\n+\twhile (o->alt_odb_list) {\n+\t\tfree_alt_odb(o->alt_odb_list);\n+\t\to->alt_odb_list = o->alt_odb_list->next;\n+\t}\n+}\n+\n void raw_object_store_clear(struct raw_object_store *o)\n {\n \tFREE_AND_NULL(o->objectdir);\n+\n+\tfree_alt_odbs(o);\n+\to->alt_odb_tail = NULL;\n }\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340119","messageId":"20180224004754.129721-6-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 05/27] object-store: move packed_git and packed_git_mru to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:32Z","receivedAt":"2018-02-24T00:48:15Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a process with multiple repositories open, packfile accessors\nshould be associated to a single repository and not shared globally.\nMove packed_git and packed_git_mru into the_repository and adjust\ncallers to reflect this.\n\nPatch generated by\n\n 1. Moving the struct packed_git declaration to object-store.h\n and packed_git, packed_git_mru globals to struct object_store.\n\n 2. Apply the following semantic patch to adjust callers:\n @@ @@\n - packed_git\n + the_repository->objects.packed_git\n\n @@ @@\n - packed_git_mru\n + the_repository->objects.packed_git_mru\n\n 3. Applying line wrapping fixes from \"make style\" to break the\n resulting long lines.\n\n 4. Adding missing #includes of repository.h where needed.\n\n 5. As the packfiles are now owned by an objectstore, which is ephemeral\n unlike globals, we introduce memory leaks. So address them in\n raw_object_store_clear(). Defer freeing packed_git to the next\n patch due to the size of this patch.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/count-objects.c | 5 +++--\n builtin/fsck.c | 6 ++++--\n builtin/gc.c | 3 ++-\n builtin/pack-objects.c | 20 +++++++++++---------\n builtin/pack-redundant.c | 5 +++--\n cache.h | 29 -----------------------------\n fast-import.c | 7 +++++--\n http-backend.c | 5 +++--\n object-store.h | 31 ++++++++++++++++++++++++++++++-\n object.c | 6 ++++++\n pack-bitmap.c | 3 ++-\n packfile.c | 39 ++++++++++++++++++++-------------------\n repository.c | 4 +++-\n server-info.c | 5 +++--\n sha1_name.c | 5 +++--\n 15 files changed, 98 insertions(+), 75 deletions(-)\n\ndiff --git a/builtin/count-objects.c b/builtin/count-objects.c\nindex 33343818c83..6aab8d08fcc 100644\n--- a/builtin/count-objects.c\n+++ b/builtin/count-objects.c\n@@ -7,6 +7,7 @@\n #include \"cache.h\"\n #include \"config.h\"\n #include \"dir.h\"\n+#include \"repository.h\"\n #include \"builtin.h\"\n #include \"parse-options.h\"\n #include \"quote.h\"\n@@ -120,9 +121,9 @@ int cmd_count_objects(int argc, const char **argv, const char *prefix)\n \t\tstruct strbuf loose_buf = STRBUF_INIT;\n \t\tstruct strbuf pack_buf = STRBUF_INIT;\n \t\tstruct strbuf garbage_buf = STRBUF_INIT;\n-\t\tif (!packed_git)\n+\t\tif (!the_repository->objects.packed_git)\n \t\t\tprepare_packed_git();\n-\t\tfor (p = packed_git; p; p = p->next) {\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t\tif (!p->pack_local)\n \t\t\t\tcontinue;\n \t\t\tif (open_pack_index(p))\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 78edcb75c45..c3a339193c8 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -735,7 +735,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t\t\tprepare_packed_git();\n \n \t\t\tif (show_progress) {\n-\t\t\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\t\tfor (p = the_repository->objects.packed_git; p;\n+\t\t\t\t p = p->next) {\n \t\t\t\t\tif (open_pack_index(p))\n \t\t\t\t\t\tcontinue;\n \t\t\t\t\ttotal += p->num_objects;\n@@ -743,7 +744,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \n \t\t\t\tprogress = start_progress(_(\"Checking objects\"), total);\n \t\t\t}\n-\t\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\tfor (p = the_repository->objects.packed_git; p;\n+\t\t\t p = p->next) {\n \t\t\t\t/* verify gives error messages itself */\n \t\t\t\tif (verify_pack(p, fsck_obj_buffer,\n \t\t\t\t\t\tprogress, count))\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 77fa720bd0b..59bf5c2ea0b 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -11,6 +11,7 @@\n */\n \n #include \"builtin.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n #include \"tempfile.h\"\n #include \"lockfile.h\"\n@@ -173,7 +174,7 @@ static int too_many_packs(void)\n \t\treturn 0;\n \n \tprepare_packed_git();\n-\tfor (cnt = 0, p = packed_git; p; p = p->next) {\n+\tfor (cnt = 0, p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (!p->pack_local)\n \t\t\tcontinue;\n \t\tif (p->pack_keep)\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 5c674b2843c..a64005760be 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1,5 +1,6 @@\n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n #include \"attr.h\"\n #include \"object.h\"\n@@ -1025,8 +1026,7 @@ static int want_object_in_pack(const struct object_id *oid,\n \t\tif (want != -1)\n \t\t\treturn want;\n \t}\n-\n-\tlist_for_each(pos, &packed_git_mru) {\n+\tlist_for_each(pos, &the_repository->objects.packed_git_mru) {\n \t\tstruct packed_git *p = list_entry(pos, struct packed_git, mru);\n \t\toff_t offset;\n \n@@ -1044,7 +1044,8 @@ static int want_object_in_pack(const struct object_id *oid,\n \t\t\t}\n \t\t\twant = want_found_object(exclude, p);\n \t\t\tif (!exclude && want > 0)\n-\t\t\t\tlist_move(&p->mru, &packed_git_mru);\n+\t\t\t\tlist_move(&p->mru,\n+\t\t\t\t\t &the_repository->objects.packed_git_mru);\n \t\t\tif (want != -1)\n \t\t\t\treturn want;\n \t\t}\n@@ -2673,7 +2674,7 @@ static void add_objects_in_unpacked_packs(struct rev_info *revs)\n \n \tmemset(&in_pack, 0, sizeof(in_pack));\n \n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tstruct object_id oid;\n \t\tstruct object *o;\n \n@@ -2736,7 +2737,8 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)\n \tstatic struct packed_git *last_found = (void *)1;\n \tstruct packed_git *p;\n \n-\tp = (last_found != (void *)1) ? last_found : packed_git;\n+\tp = (last_found != (void *)1) ? last_found :\n+\t\t\t\t\tthe_repository->objects.packed_git;\n \n \twhile (p) {\n \t\tif ((!p->pack_local || p->pack_keep) &&\n@@ -2745,7 +2747,7 @@ static int has_sha1_pack_kept_or_nonlocal(const struct object_id *oid)\n \t\t\treturn 1;\n \t\t}\n \t\tif (p == last_found)\n-\t\t\tp = packed_git;\n+\t\t\tp = the_repository->objects.packed_git;\n \t\telse\n \t\t\tp = p->next;\n \t\tif (p == last_found)\n@@ -2781,7 +2783,7 @@ static void loosen_unused_packed_objects(struct rev_info *revs)\n \tuint32_t i;\n \tstruct object_id oid;\n \n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (!p->pack_local || p->pack_keep)\n \t\t\tcontinue;\n \n@@ -3152,7 +3154,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \tprepare_packed_git();\n \tif (ignore_packed_keep) {\n \t\tstruct packed_git *p;\n-\t\tfor (p = packed_git; p; p = p->next)\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\t\tif (p->pack_local && p->pack_keep)\n \t\t\t\tbreak;\n \t\tif (!p) /* no keep-able packs found */\n@@ -3165,7 +3167,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)\n \t\t * also covers non-local objects\n \t\t */\n \t\tstruct packed_git *p;\n-\t\tfor (p = packed_git; p; p = p->next) {\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t\tif (!p->pack_local) {\n \t\t\t\thave_non_local_packs = 1;\n \t\t\t\tbreak;\ndiff --git a/builtin/pack-redundant.c b/builtin/pack-redundant.c\nindex aaa81363222..a196fae79d4 100644\n--- a/builtin/pack-redundant.c\n+++ b/builtin/pack-redundant.c\n@@ -7,6 +7,7 @@\n */\n \n #include \"builtin.h\"\n+#include \"repository.h\"\n #include \"packfile.h\"\n \n #define BLKSIZE 512\n@@ -571,7 +572,7 @@ static struct pack_list * add_pack(struct packed_git *p)\n \n static struct pack_list * add_pack_file(const char *filename)\n {\n-\tstruct packed_git *p = packed_git;\n+\tstruct packed_git *p = the_repository->objects.packed_git;\n \n \tif (strlen(filename) < 40)\n \t\tdie(\"Bad pack filename: %s\", filename);\n@@ -586,7 +587,7 @@ static struct pack_list * add_pack_file(const char *filename)\n \n static void load_all(void)\n {\n-\tstruct packed_git *p = packed_git;\n+\tstruct packed_git *p = the_repository->objects.packed_git;\n \n \twhile (p) {\n \t\tadd_pack(p);\ndiff --git a/cache.h b/cache.h\nindex 99eb2bdb893..63889acb591 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1569,35 +1569,6 @@ struct pack_window {\n \tunsigned int inuse_cnt;\n };\n \n-extern struct packed_git {\n-\tstruct packed_git *next;\n-\tstruct list_head mru;\n-\tstruct pack_window *windows;\n-\toff_t pack_size;\n-\tconst void *index_data;\n-\tsize_t index_size;\n-\tuint32_t num_objects;\n-\tuint32_t num_bad_objects;\n-\tunsigned char *bad_object_sha1;\n-\tint index_version;\n-\ttime_t mtime;\n-\tint pack_fd;\n-\tunsigned pack_local:1,\n-\t\t pack_keep:1,\n-\t\t freshened:1,\n-\t\t do_not_close:1,\n-\t\t pack_promisor:1;\n-\tunsigned char sha1[20];\n-\tstruct revindex_entry *revindex;\n-\t/* something like \".git/objects/pack/xxxxx.pack\" */\n-\tchar pack_name[FLEX_ARRAY]; /* more */\n-} *packed_git;\n-\n-/*\n- * A most-recently-used ordered version of the packed_git list.\n- */\n-extern struct list_head packed_git_mru;\n-\n struct pack_entry {\n \toff_t offset;\n \tunsigned char sha1[20];\ndiff --git a/fast-import.c b/fast-import.c\nindex 0f818cd9328..3d7002d1cd6 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -154,6 +154,7 @@ Format of STDIN stream:\n \n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n #include \"lockfile.h\"\n #include \"object.h\"\n@@ -1110,7 +1111,8 @@ static int store_object(\n \tif (e->idx.offset) {\n \t\tduplicate_count_by_type[type]++;\n \t\treturn 1;\n-\t} else if (find_sha1_pack(oid.hash, packed_git)) {\n+\t} else if (find_sha1_pack(oid.hash,\n+\t\t\t\t the_repository->objects.packed_git)) {\n \t\te->type = type;\n \t\te->pack_id = MAX_PACK_ID;\n \t\te->idx.offset = 1; /* just not zero! */\n@@ -1307,7 +1309,8 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)\n \t\tduplicate_count_by_type[OBJ_BLOB]++;\n \t\ttruncate_pack(&checkpoint);\n \n-\t} else if (find_sha1_pack(oid.hash, packed_git)) {\n+\t} else if (find_sha1_pack(oid.hash,\n+\t\t\t\t the_repository->objects.packed_git)) {\n \t\te->type = OBJ_BLOB;\n \t\te->pack_id = MAX_PACK_ID;\n \t\te->idx.offset = 1; /* just not zero! */\ndiff --git a/http-backend.c b/http-backend.c\nindex f3dc218b2a3..85b6a97923f 100644\n--- a/http-backend.c\n+++ b/http-backend.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"config.h\"\n+#include \"repository.h\"\n #include \"refs.h\"\n #include \"pkt-line.h\"\n #include \"object.h\"\n@@ -518,13 +519,13 @@ static void get_info_packs(struct strbuf *hdr, char *arg)\n \n \tselect_getanyfile(hdr);\n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (p->pack_local)\n \t\t\tcnt++;\n \t}\n \n \tstrbuf_grow(&buf, cnt * 53 + 2);\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (p->pack_local)\n \t\t\tstrbuf_addf(&buf, \"P %s\\n\", p->pack_name + objdirlen + 6);\n \t}\ndiff --git a/object-store.h b/object-store.h\nindex e78eea1dde3..4f768465a14 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -52,6 +52,30 @@ void add_to_alternates_memory(const char *dir);\n */\n struct strbuf *alt_scratch_buf(struct alternate_object_database *alt);\n \n+struct packed_git {\n+\tstruct packed_git *next;\n+\tstruct list_head mru;\n+\tstruct pack_window *windows;\n+\toff_t pack_size;\n+\tconst void *index_data;\n+\tsize_t index_size;\n+\tuint32_t num_objects;\n+\tuint32_t num_bad_objects;\n+\tunsigned char *bad_object_sha1;\n+\tint index_version;\n+\ttime_t mtime;\n+\tint pack_fd;\n+\tunsigned pack_local:1,\n+\t\t pack_keep:1,\n+\t\t freshened:1,\n+\t\t do_not_close:1,\n+\t\t pack_promisor:1;\n+\tunsigned char sha1[20];\n+\tstruct revindex_entry *revindex;\n+\t/* something like \".git/objects/pack/xxxxx.pack\" */\n+\tchar pack_name[FLEX_ARRAY]; /* more */\n+};\n+\n struct raw_object_store {\n \t/*\n \t * Path to the repository's object store.\n@@ -59,10 +83,15 @@ struct raw_object_store {\n \t */\n \tchar *objectdir;\n \n+\tstruct packed_git *packed_git;\n+\t/* A most-recently-used ordered version of the packed_git list. */\n+\tstruct list_head packed_git_mru;\n+\n \tstruct alternate_object_database *alt_odb_list;\n \tstruct alternate_object_database **alt_odb_tail;\n };\n-#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }\n+\n+#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/object.c b/object.c\nindex 920dc4463fa..367441efa94 100644\n--- a/object.c\n+++ b/object.c\n@@ -466,4 +466,10 @@ void raw_object_store_clear(struct raw_object_store *o)\n \n \tfree_alt_odbs(o);\n \to->alt_odb_tail = NULL;\n+\n+\tINIT_LIST_HEAD(&o->packed_git_mru);\n+\t/*\n+\t * TODO: call close_all_packs once migrated to\n+\t * take an object store argument\n+\t */\n }\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 9270983e5f5..bbede32589a 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -10,6 +10,7 @@\n #include \"pack-revindex.h\"\n #include \"pack-objects.h\"\n #include \"packfile.h\"\n+#include \"repository.h\"\n \n /*\n * An entry on the bitmap index, representing the bitmap for a given\n@@ -335,7 +336,7 @@ static int open_pack_bitmap(void)\n \tassert(!bitmap_git.map && !bitmap_git.loaded);\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (open_pack_bitmap_1(p) == 0)\n \t\t\tret = 0;\n \t}\ndiff --git a/packfile.c b/packfile.c\nindex 216ea836ee3..59210deaaf7 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -45,8 +45,6 @@ static unsigned int pack_open_fds;\n static unsigned int pack_max_fds;\n static size_t peak_pack_mapped;\n static size_t pack_mapped;\n-struct packed_git *packed_git;\n-LIST_HEAD(packed_git_mru);\n \n #define SZ_FMT PRIuMAX\n static inline uintmax_t sz_fmt(size_t s) { return s; }\n@@ -246,7 +244,7 @@ static int unuse_one_window(struct packed_git *current)\n \n \tif (current)\n \t\tscan_windows(current, &lru_p, &lru_w, &lru_l);\n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tscan_windows(p, &lru_p, &lru_w, &lru_l);\n \tif (lru_p) {\n \t\tmunmap(lru_w->base, lru_w->len);\n@@ -316,7 +314,7 @@ void close_all_packs(void)\n {\n \tstruct packed_git *p;\n \n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tif (p->do_not_close)\n \t\t\tdie(\"BUG: want to close pack marked 'do-not-close'\");\n \t\telse\n@@ -384,7 +382,7 @@ static int close_one_pack(void)\n \tstruct pack_window *mru_w = NULL;\n \tint accept_windows_inuse = 1;\n \n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (p->pack_fd == -1)\n \t\t\tcontinue;\n \t\tfind_lru_pack(p, &lru_p, &mru_w, &accept_windows_inuse);\n@@ -686,8 +684,8 @@ void install_packed_git(struct packed_git *pack)\n \tif (pack->pack_fd != -1)\n \t\tpack_open_fds++;\n \n-\tpack->next = packed_git;\n-\tpacked_git = pack;\n+\tpack->next = the_repository->objects.packed_git;\n+\tthe_repository->objects.packed_git = pack;\n }\n \n void (*report_garbage)(unsigned seen_bits, const char *path);\n@@ -769,7 +767,8 @@ static void prepare_packed_git_one(char *objdir, int local)\n \t\tbase_len = path.len;\n \t\tif (strip_suffix_mem(path.buf, &base_len, \".idx\")) {\n \t\t\t/* Don't reopen a pack we already have. */\n-\t\t\tfor (p = packed_git; p; p = p->next) {\n+\t\t\tfor (p = the_repository->objects.packed_git; p;\n+\t\t\t p = p->next) {\n \t\t\t\tsize_t len;\n \t\t\t\tif (strip_suffix(p->pack_name, \".pack\", &len) &&\n \t\t\t\t len == base_len &&\n@@ -820,7 +819,7 @@ unsigned long approximate_object_count(void)\n \n \t\tprepare_packed_git();\n \t\tcount = 0;\n-\t\tfor (p = packed_git; p; p = p->next) {\n+\t\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t\tif (open_pack_index(p))\n \t\t\t\tcontinue;\n \t\t\tcount += p->num_objects;\n@@ -869,18 +868,19 @@ static int sort_pack(const void *a_, const void *b_)\n \n static void rearrange_packed_git(void)\n {\n-\tpacked_git = llist_mergesort(packed_git, get_next_packed_git,\n-\t\t\t\t set_next_packed_git, sort_pack);\n+\tthe_repository->objects.packed_git = llist_mergesort(\n+\t\tthe_repository->objects.packed_git, get_next_packed_git,\n+\t\tset_next_packed_git, sort_pack);\n }\n \n static void prepare_packed_git_mru(void)\n {\n \tstruct packed_git *p;\n \n-\tINIT_LIST_HEAD(&packed_git_mru);\n+\tINIT_LIST_HEAD(&the_repository->objects.packed_git_mru);\n \n-\tfor (p = packed_git; p; p = p->next)\n-\t\tlist_add_tail(&p->mru, &packed_git_mru);\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n+\t\tlist_add_tail(&p->mru, &the_repository->objects.packed_git_mru);\n }\n \n static int prepare_packed_git_run_once = 0;\n@@ -1014,7 +1014,7 @@ const struct packed_git *has_packed_and_bad(const unsigned char *sha1)\n \tstruct packed_git *p;\n \tunsigned i;\n \n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tfor (i = 0; i < p->num_bad_objects; i++)\n \t\t\tif (!hashcmp(sha1, p->bad_object_sha1 + 20 * i))\n \t\t\t\treturn p;\n@@ -1845,13 +1845,14 @@ int find_pack_entry(const unsigned char *sha1, struct pack_entry *e)\n \tstruct list_head *pos;\n \n \tprepare_packed_git();\n-\tif (!packed_git)\n+\tif (!the_repository->objects.packed_git)\n \t\treturn 0;\n \n-\tlist_for_each(pos, &packed_git_mru) {\n+\tlist_for_each(pos, &the_repository->objects.packed_git_mru) {\n \t\tstruct packed_git *p = list_entry(pos, struct packed_git, mru);\n \t\tif (fill_pack_entry(sha1, e, p)) {\n-\t\t\tlist_move(&p->mru, &packed_git_mru);\n+\t\t\tlist_move(&p->mru,\n+\t\t\t\t &the_repository->objects.packed_git_mru);\n \t\t\treturn 1;\n \t\t}\n \t}\n@@ -1898,7 +1899,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data, unsigned flags)\n \tint pack_errors = 0;\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)\n \t\t\tcontinue;\n \t\tif ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&\ndiff --git a/repository.c b/repository.c\nindex fb65e8072d5..a069b1b6404 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -7,7 +7,7 @@\n /* The main repository */\n static struct repository the_repo = {\n \tNULL, NULL,\n-\tRAW_OBJECT_STORE_INIT,\n+\tRAW_OBJECT_STORE_INIT(the_repo.objects),\n \tNULL, NULL, NULL,\n \tNULL, NULL, NULL,\n \t&the_index,\n@@ -143,6 +143,8 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)\n \n \trepo->ignore_env = 1;\n \n+\tINIT_LIST_HEAD(&repo->objects.packed_git_mru);\n+\n \tif (repo_init_gitdir(repo, gitdir))\n \t\tgoto error;\n \ndiff --git a/server-info.c b/server-info.c\nindex 26a6c20b7d4..c369ac27cbc 100644\n--- a/server-info.c\n+++ b/server-info.c\n@@ -1,4 +1,5 @@\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"refs.h\"\n #include \"object.h\"\n #include \"commit.h\"\n@@ -200,7 +201,7 @@ static void init_pack_info(const char *infofile, int force)\n \tobjdirlen = strlen(objdir);\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next) {\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next) {\n \t\t/* we ignore things on alternate path since they are\n \t\t * not available to the pullers in general.\n \t\t */\n@@ -210,7 +211,7 @@ static void init_pack_info(const char *infofile, int force)\n \t}\n \tnum_pack = i;\n \tinfo = xcalloc(num_pack, sizeof(struct pack_info *));\n-\tfor (i = 0, p = packed_git; p; p = p->next) {\n+\tfor (i = 0, p = the_repository->objects.packed_git; p; p = p->next) {\n \t\tif (!p->pack_local)\n \t\t\tcontinue;\n \t\tinfo[i] = xcalloc(1, sizeof(struct pack_info));\ndiff --git a/sha1_name.c b/sha1_name.c\nindex 957ce25680b..016c883b5c7 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -196,7 +196,8 @@ static void find_short_packed_object(struct disambiguate_state *ds)\n \tstruct packed_git *p;\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p && !ds->ambiguous; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p && !ds->ambiguous;\n+\t p = p->next)\n \t\tunique_in_pack(p, ds);\n }\n \n@@ -566,7 +567,7 @@ static void find_abbrev_len_packed(struct min_abbrev_data *mad)\n \tstruct packed_git *p;\n \n \tprepare_packed_git();\n-\tfor (p = packed_git; p; p = p->next)\n+\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n \t\tfind_abbrev_len_for_pack(p, mad);\n }\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340120","messageId":"20180224004754.129721-9-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 08/27] pack: move approximate object count to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:35Z","receivedAt":"2018-02-24T00:48:19Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"The approximate_object_count() function maintains a rough count of\nobjects in a repository to estimate how long object name abbreviates\nshould be. Object names are scoped to a repository and the\nappropriate length may differ by repository, so the object count\nshould not be global.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 10 +++++++++-\n packfile.c | 11 +++++------\n 2 files changed, 14 insertions(+), 7 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex bd6441e525f..31ca4d3f340 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -90,6 +90,14 @@ struct raw_object_store {\n \tstruct alternate_object_database *alt_odb_list;\n \tstruct alternate_object_database **alt_odb_tail;\n \n+\t/*\n+\t * A fast, rough count of the number of objects in the repository.\n+\t * These two fields are not meant for direct access. Use\n+\t * approximate_object_count() instead.\n+\t */\n+\tunsigned long approximate_object_count;\n+\tunsigned approximate_object_count_valid : 1;\n+\n \t/*\n \t * Whether packed_git has already been populated with this repository's\n \t * packs.\n@@ -97,7 +105,7 @@ struct raw_object_store {\n \tunsigned packed_git_initialized : 1;\n };\n \n-#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0 }\n+#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0, 0, 0 }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/packfile.c b/packfile.c\nindex 3ee349ab1bd..8d7b1dd5f9c 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -802,8 +802,6 @@ static void prepare_packed_git_one(char *objdir, int local)\n \tstrbuf_release(&path);\n }\n \n-static int approximate_object_count_valid;\n-\n /*\n * Give a fast, rough count of the number of objects in the repository. This\n * ignores loose objects completely. If you have a lot of them, then either\n@@ -813,8 +811,8 @@ static int approximate_object_count_valid;\n */\n unsigned long approximate_object_count(void)\n {\n-\tstatic unsigned long count;\n-\tif (!approximate_object_count_valid) {\n+\tif (!the_repository->objects.approximate_object_count_valid) {\n+\t\tunsigned long count;\n \t\tstruct packed_git *p;\n \n \t\tprepare_packed_git();\n@@ -824,8 +822,9 @@ unsigned long approximate_object_count(void)\n \t\t\t\tcontinue;\n \t\t\tcount += p->num_objects;\n \t\t}\n+\t\tthe_repository->objects.approximate_object_count = count;\n \t}\n-\treturn count;\n+\treturn the_repository->objects.approximate_object_count;\n }\n \n static void *get_next_packed_git(const void *p)\n@@ -900,7 +899,7 @@ void prepare_packed_git(void)\n \n void reprepare_packed_git(void)\n {\n-\tapproximate_object_count_valid = 0;\n+\tthe_repository->objects.approximate_object_count_valid = 0;\n \tthe_repository->objects.packed_git_initialized = 0;\n \tprepare_packed_git();\n }\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340121","messageId":"20180224004754.129721-7-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 06/27] object-store: close all packs upon clearing the object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:33Z","receivedAt":"2018-02-24T00:48:21Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n builtin/am.c | 2 +-\n builtin/clone.c | 2 +-\n builtin/fetch.c | 2 +-\n builtin/merge.c | 2 +-\n builtin/receive-pack.c | 2 +-\n object.c | 7 +++----\n packfile.c | 4 ++--\n packfile.h | 2 +-\n 8 files changed, 11 insertions(+), 12 deletions(-)\n\ndiff --git a/builtin/am.c b/builtin/am.c\nindex 6661edc162b..fc1d9b43c51 100644\n--- a/builtin/am.c\n+++ b/builtin/am.c\n@@ -1859,7 +1859,7 @@ static void am_run(struct am_state *state, int resume)\n \t */\n \tif (!state->rebasing) {\n \t\tam_destroy(state);\n-\t\tclose_all_packs();\n+\t\tclose_all_packs(&the_repository->objects);\n \t\trun_command_v_opt(argv_gc_auto, RUN_GIT_CMD);\n \t}\n }\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 101c27a593f..13cfaa6488a 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -1217,7 +1217,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)\n \ttransport_disconnect(transport);\n \n \tif (option_dissociate) {\n-\t\tclose_all_packs();\n+\t\tclose_all_packs(&the_repository->objects);\n \t\tdissociate_from_references();\n \t}\n \ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 8ee998ea2ee..4d72efca781 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -1478,7 +1478,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)\n \n \tstring_list_clear(&list, 0);\n \n-\tclose_all_packs();\n+\tclose_all_packs(&the_repository->objects);\n \n \targv_array_pushl(&argv_gc_auto, \"gc\", \"--auto\", NULL);\n \tif (verbosity < 0)\ndiff --git a/builtin/merge.c b/builtin/merge.c\nindex 92ba99a1a5e..0af0c53a632 100644\n--- a/builtin/merge.c\n+++ b/builtin/merge.c\n@@ -411,7 +411,7 @@ static void finish(struct commit *head_commit,\n \t\t\t * We ignore errors in 'gc --auto', since the\n \t\t\t * user should see them.\n \t\t\t */\n-\t\t\tclose_all_packs();\n+\t\t\tclose_all_packs(&the_repository->objects);\n \t\t\trun_command_v_opt(argv_gc_auto, RUN_GIT_CMD);\n \t\t}\n \t}\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex 75e7f18acef..8e25aae54c8 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -2027,7 +2027,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)\n \t\t\tproc.git_cmd = 1;\n \t\t\tproc.argv = argv_gc_auto;\n \n-\t\t\tclose_all_packs();\n+\t\t\tclose_all_packs(&the_repository->objects);\n \t\t\tif (!start_command(&proc)) {\n \t\t\t\tif (use_sideband)\n \t\t\t\t\tcopy_to_sideband(proc.err, -1, NULL);\ndiff --git a/object.c b/object.c\nindex 367441efa94..a7c238339bd 100644\n--- a/object.c\n+++ b/object.c\n@@ -4,6 +4,7 @@\n #include \"tree.h\"\n #include \"commit.h\"\n #include \"tag.h\"\n+#include \"packfile.h\"\n \n static struct object **obj_hash;\n static int nr_objs, obj_hash_size;\n@@ -468,8 +469,6 @@ void raw_object_store_clear(struct raw_object_store *o)\n \to->alt_odb_tail = NULL;\n \n \tINIT_LIST_HEAD(&o->packed_git_mru);\n-\t/*\n-\t * TODO: call close_all_packs once migrated to\n-\t * take an object store argument\n-\t */\n+\tclose_all_packs(o);\n+\to->packed_git = NULL;\n }\ndiff --git a/packfile.c b/packfile.c\nindex 59210deaaf7..65d9a4f6c61 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -310,11 +310,11 @@ static void close_pack(struct packed_git *p)\n \tclose_pack_index(p);\n }\n \n-void close_all_packs(void)\n+void close_all_packs(struct raw_object_store *o)\n {\n \tstruct packed_git *p;\n \n-\tfor (p = the_repository->objects.packed_git; p; p = p->next)\n+\tfor (p = o->packed_git; p; p = p->next)\n \t\tif (p->do_not_close)\n \t\t\tdie(\"BUG: want to close pack marked 'do-not-close'\");\n \t\telse\ndiff --git a/packfile.h b/packfile.h\nindex a7fca598d67..6a2c57045ca 100644\n--- a/packfile.h\n+++ b/packfile.h\n@@ -63,7 +63,7 @@ extern void close_pack_index(struct packed_git *);\n \n extern unsigned char *use_pack(struct packed_git *, struct pack_window **, off_t, unsigned long *);\n extern void close_pack_windows(struct packed_git *);\n-extern void close_all_packs(void);\n+extern void close_all_packs(struct raw_object_store *o);\n extern void unuse_pack(struct pack_window **);\n extern void clear_delta_base_cache(void);\n extern struct packed_git *add_packed_git(const char *path, size_t path_len, int local);\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340122","messageId":"20180224004754.129721-4-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 03/27] object-store: move alt_odb_list and alt_odb_tail to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:30Z","receivedAt":"2018-02-24T00:48:24Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"In a process with multiple repositories open, alternates should be\nassociated to a single repository and not shared globally. Move\nalt_odb_list and alt_odb_tail into the_repository and adjust callers\nto reflect this.\n\nNow that the alternative object data base is per repository, we're\nleaking its memory upon freeing a repository. The next patch plugs\nthis hole.\n\nNo functional change intended.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/fsck.c | 4 +++-\n object-store.h | 9 ++++++---\n packfile.c | 3 ++-\n sha1_file.c | 25 ++++++++++++-------------\n sha1_name.c | 3 ++-\n 5 files changed, 25 insertions(+), 19 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex 9981db22637..78edcb75c45 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -1,5 +1,6 @@\n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"config.h\"\n #include \"commit.h\"\n #include \"tree.h\"\n@@ -722,7 +723,8 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t\tfsck_object_dir(get_object_directory());\n \n \t\tprepare_alt_odb();\n-\t\tfor (alt = alt_odb_list; alt; alt = alt->next)\n+\t\tfor (alt = the_repository->objects.alt_odb_list;\n+\t\t alt; alt = alt->next)\n \t\t\tfsck_object_dir(alt->path);\n \n \t\tif (check_full) {\ndiff --git a/object-store.h b/object-store.h\nindex 5678aa1136c..e78eea1dde3 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -1,7 +1,7 @@\n #ifndef OBJECT_STORE_H\n #define OBJECT_STORE_H\n \n-extern struct alternate_object_database {\n+struct alternate_object_database {\n \tstruct alternate_object_database *next;\n \n \t/* see alt_scratch_buf() */\n@@ -19,7 +19,7 @@ extern struct alternate_object_database {\n \tstruct oid_array loose_objects_cache;\n \n \tchar path[FLEX_ARRAY];\n-} *alt_odb_list;\n+};\n void prepare_alt_odb(void);\n char *compute_alternate_path(const char *path, struct strbuf *err);\n typedef int alt_odb_fn(struct alternate_object_database *, void *);\n@@ -58,8 +58,11 @@ struct raw_object_store {\n \t * Cannot be NULL after initialization.\n \t */\n \tchar *objectdir;\n+\n+\tstruct alternate_object_database *alt_odb_list;\n+\tstruct alternate_object_database **alt_odb_tail;\n };\n-#define RAW_OBJECT_STORE_INIT { NULL }\n+#define RAW_OBJECT_STORE_INIT { NULL, NULL, NULL }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/packfile.c b/packfile.c\nindex 7dbe8739d17..216ea836ee3 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -1,6 +1,7 @@\n #include \"cache.h\"\n #include \"list.h\"\n #include \"pack.h\"\n+#include \"repository.h\"\n #include \"dir.h\"\n #include \"mergesort.h\"\n #include \"packfile.h\"\n@@ -891,7 +892,7 @@ void prepare_packed_git(void)\n \t\treturn;\n \tprepare_packed_git_one(get_object_directory(), 1);\n \tprepare_alt_odb();\n-\tfor (alt = alt_odb_list; alt; alt = alt->next)\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)\n \t\tprepare_packed_git_one(alt->path, 0);\n \trearrange_packed_git();\n \tprepare_packed_git_mru();\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 826d7a0ae37..ac25146076e 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -22,6 +22,7 @@\n #include \"pack-revindex.h\"\n #include \"sha1-lookup.h\"\n #include \"bulk-checkin.h\"\n+#include \"repository.h\"\n #include \"streaming.h\"\n #include \"dir.h\"\n #include \"list.h\"\n@@ -340,9 +341,6 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,\n \treturn buf->buf;\n }\n \n-struct alternate_object_database *alt_odb_list;\n-static struct alternate_object_database **alt_odb_tail;\n-\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\n@@ -362,7 +360,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)\n \t * Prevent the common mistake of listing the same\n \t * thing twice, or object directory itself.\n \t */\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\tif (!fspathcmp(path->buf, alt->path))\n \t\t\treturn 0;\n \t}\n@@ -422,8 +420,8 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,\n \tent = alloc_alt_odb(pathbuf.buf);\n \n \t/* add the alternate entry */\n-\t*alt_odb_tail = ent;\n-\talt_odb_tail = &(ent->next);\n+\t*the_repository->objects.alt_odb_tail = ent;\n+\tthe_repository->objects.alt_odb_tail = &(ent->next);\n \tent->next = NULL;\n \n \t/* recursively add alternates */\n@@ -557,7 +555,7 @@ void add_to_alternates_file(const char *reference)\n \t\tfprintf_or_die(out, \"%s\\n\", reference);\n \t\tif (commit_lock_file(&lock))\n \t\t\tdie_errno(\"unable to move new alternates file into place\");\n-\t\tif (alt_odb_tail)\n+\t\tif (the_repository->objects.alt_odb_tail)\n \t\t\tlink_alt_odb_entries(reference, '\\n', NULL, 0);\n \t}\n \tfree(alts);\n@@ -655,7 +653,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \tint r = 0;\n \n \tprepare_alt_odb();\n-\tfor (ent = alt_odb_list; ent; ent = ent->next) {\n+\tfor (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {\n \t\tr = fn(ent, cb);\n \t\tif (r)\n \t\t\tbreak;\n@@ -667,12 +665,13 @@ void prepare_alt_odb(void)\n {\n \tconst char *alt;\n \n-\tif (alt_odb_tail)\n+\tif (the_repository->objects.alt_odb_tail)\n \t\treturn;\n \n \talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n \n-\talt_odb_tail = &alt_odb_list;\n+\tthe_repository->objects.alt_odb_tail =\n+\t\t\t&the_repository->objects.alt_odb_list;\n \tlink_alt_odb_entries(alt, PATH_SEP, NULL, 0);\n \n \tread_info_alternates(get_object_directory(), 0);\n@@ -716,7 +715,7 @@ static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)\n {\n \tstruct alternate_object_database *alt;\n \tprepare_alt_odb();\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\tconst char *path = alt_sha1_path(alt, sha1);\n \t\tif (check_and_freshen_file(path, freshen))\n \t\t\treturn 1;\n@@ -876,7 +875,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n \n \tprepare_alt_odb();\n \terrno = ENOENT;\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tif (!lstat(*path, st))\n \t\t\treturn 0;\n@@ -906,7 +905,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \tmost_interesting_errno = errno;\n \n \tprepare_alt_odb();\n-\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tfd = git_open(*path);\n \t\tif (fd >= 0)\ndiff --git a/sha1_name.c b/sha1_name.c\nindex 611c7d24dde..957ce25680b 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -10,6 +10,7 @@\n #include \"dir.h\"\n #include \"sha1-array.h\"\n #include \"packfile.h\"\n+#include \"repository.h\"\n \n static int get_oid_oneline(const char *, struct object_id *, struct commit_list *);\n \n@@ -104,7 +105,7 @@ static void find_short_object_filename(struct disambiguate_state *ds)\n \t\t */\n \t\tfakeent = alloc_alt_odb(get_object_directory());\n \t}\n-\tfakeent->next = alt_odb_list;\n+\tfakeent->next = the_repository->objects.alt_odb_list;\n \n \tfor (alt = fakeent; alt && !ds->ambiguous; alt = alt->next) {\n \t\tint pos;\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340123","messageId":"20180224004754.129721-16-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 15/27] sha1_file: allow prepare_alt_odb to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:42Z","receivedAt":"2018-02-24T00:48:28Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \n---\n object-store.h | 3 +--\n sha1_file.c | 12 +++++-------\n 2 files changed, 6 insertions(+), 9 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 08cd48ade11..24b9a750aef 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -24,8 +24,7 @@ struct alternate_object_database {\n \t */\n \tchar path[FLEX_ARRAY];\n };\n-#define prepare_alt_odb(r) prepare_alt_odb_##r()\n-void prepare_alt_odb_the_repository(void);\n+void prepare_alt_odb(struct repository *r);\n char *compute_alternate_path(const char *path, struct strbuf *err);\n typedef int alt_odb_fn(struct alternate_object_database *, void *);\n int foreach_alt_odb(alt_odb_fn, void*);\ndiff --git a/sha1_file.c b/sha1_file.c\nindex ab42d430b8a..03b9bbe8bb7 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -671,21 +671,19 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \treturn r;\n }\n \n-void prepare_alt_odb_the_repository(void)\n+void prepare_alt_odb(struct repository *r)\n {\n \tconst char *alt;\n \n-\tif (the_repository->objects.alt_odb_tail)\n+\tif (r->objects.alt_odb_tail)\n \t\treturn;\n \n \talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n \n-\tthe_repository->objects.alt_odb_tail =\n-\t\t\t&the_repository->objects.alt_odb_list;\n-\tlink_alt_odb_entries(the_repository, alt,\n-\t\t\t PATH_SEP, NULL, 0);\n+\tr->objects.alt_odb_tail = &r->objects.alt_odb_list;\n+\tlink_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);\n \n-\tread_info_alternates(the_repository, get_object_directory(), 0);\n+\tread_info_alternates(r, r->objects.objectdir, 0);\n }\n \n /* Returns 1 if we have successfully freshened the file, 0 otherwise. */\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340124","messageId":"20180224004754.129721-20-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 19/27] sha1_file: add repository argument to map_sha1_file_1","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:46Z","receivedAt":"2018-02-24T00:48:31Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the map_sha1_file_1 caller to be\nmore specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 ++++++-----\n 1 file changed, 6 insertions(+), 5 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 7e471e0cf4b..d4af8f56814 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -933,9 +933,10 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,\n * Map the loose object at \"path\" if it is not NULL, or the path found by\n * searching for a loose object named \"sha1\".\n */\n-static void *map_sha1_file_1(const char *path,\n-\t\t\t const unsigned char *sha1,\n-\t\t\t unsigned long *size)\n+#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)\n+static void *map_sha1_file_1_the_repository(const char *path,\n+\t\t\t\t\t const unsigned char *sha1,\n+\t\t\t\t\t unsigned long *size)\n {\n \tvoid *map;\n \tint fd;\n@@ -964,7 +965,7 @@ static void *map_sha1_file_1(const char *path,\n \n void *map_sha1_file(const unsigned char *sha1, unsigned long *size)\n {\n-\treturn map_sha1_file_1(NULL, sha1, size);\n+\treturn map_sha1_file_1(the_repository, NULL, sha1, size);\n }\n \n static int unpack_sha1_short_header(git_zstream *stream,\n@@ -2200,7 +2201,7 @@ int read_loose_object(const char *path,\n \n \t*contents = NULL;\n \n-\tmap = map_sha1_file_1(path, NULL, &mapsize);\n+\tmap = map_sha1_file_1(the_repository, path, NULL, &mapsize);\n \tif (!map) {\n \t\terror_errno(\"unable to mmap %s\", path);\n \t\tgoto out;\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340125","messageId":"20180224004754.129721-17-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 16/27] sha1_file: add repository argument to sha1_file_name","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:43Z","receivedAt":"2018-02-24T00:48:32Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow sha1_file_name callers to be more\nspecific about which repository to handle. This is a small mechanical\nchange; it doesn't change the implementation to handle repositories\nother than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nWhile at it, move the declaration to object-store.h, where it should\nbe easier to find.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n cache.h | 6 ------\n http-walker.c | 3 ++-\n http.c | 5 ++---\n object-store.h | 7 +++++++\n sha1_file.c | 10 +++++-----\n 5 files changed, 16 insertions(+), 15 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 63889acb591..94d85a5c692 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -936,12 +936,6 @@ extern void check_repository_format(void);\n #define DATA_CHANGED 0x0020\n #define TYPE_CHANGED 0x0040\n \n-/*\n- * Put in `buf` the name of the file in the local object database that\n- * would be used to store a loose object with the specified sha1.\n- */\n-extern void sha1_file_name(struct strbuf *buf, const unsigned char *sha1);\n-\n /*\n * Return an abbreviated sha1 unique within this repository's object database.\n * The result will be at least `len` characters long, and will be NUL\ndiff --git a/http-walker.c b/http-walker.c\nindex 07c2b1af826..2c33969123a 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -1,4 +1,5 @@\n #include \"cache.h\"\n+#include \"repository.h\"\n #include \"commit.h\"\n #include \"walker.h\"\n #include \"http.h\"\n@@ -545,7 +546,7 @@ static int fetch_object(struct walker *walker, unsigned char *sha1)\n \t\tret = error(\"File %s has bad hash\", hex);\n \t} else if (req->rename < 0) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n-\t\tsha1_file_name(&buf, req->sha1);\n+\t\tsha1_file_name(the_repository, &buf, req->sha1);\n \t\tret = error(\"unable to write sha1 filename %s\", buf.buf);\n \t\tstrbuf_release(&buf);\n \t}\ndiff --git a/http.c b/http.c\nindex 31755023a4f..df9dbea59c5 100644\n--- a/http.c\n+++ b/http.c\n@@ -2246,7 +2246,7 @@ struct http_object_request *new_http_object_request(const char *base_url,\n \thashcpy(freq->sha1, sha1);\n \tfreq->localfile = -1;\n \n-\tsha1_file_name(&filename, sha1);\n+\tsha1_file_name(the_repository, &filename, sha1);\n \tsnprintf(freq->tmpfile, sizeof(freq->tmpfile),\n \t\t \"%s.temp\", filename.buf);\n \n@@ -2395,8 +2395,7 @@ int finish_http_object_request(struct http_object_request *freq)\n \t\tunlink_or_warn(freq->tmpfile);\n \t\treturn -1;\n \t}\n-\n-\tsha1_file_name(&filename, freq->sha1);\n+\tsha1_file_name(the_repository, &filename, freq->sha1);\n \tfreq->rename = finalize_object_file(freq->tmpfile, filename.buf);\n \tstrbuf_release(&filename);\n \ndiff --git a/object-store.h b/object-store.h\nindex 24b9a750aef..c0d1292cfc2 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -113,4 +113,11 @@ struct raw_object_store {\n \n void raw_object_store_clear(struct raw_object_store *o);\n \n+/*\n+ * Put in `buf` the name of the file in the local object database that\n+ * would be used to store a loose object with the specified sha1.\n+ */\n+#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)\n+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);\n+\n #endif /* OBJECT_STORE_H */\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 03b9bbe8bb7..8ef2f856035 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -321,7 +321,7 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)\n \t}\n }\n \n-void sha1_file_name(struct strbuf *buf, const unsigned char *sha1)\n+void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)\n {\n \tstrbuf_addstr(buf, get_object_directory());\n \tstrbuf_addch(buf, '/');\n@@ -715,7 +715,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(&buf, sha1);\n+\tsha1_file_name(the_repository, &buf, sha1);\n \n \treturn check_and_freshen_file(buf.buf, freshen);\n }\n@@ -876,7 +876,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(&buf, sha1);\n+\tsha1_file_name(the_repository, &buf, sha1);\n \t*path = buf.buf;\n \n \tif (!lstat(*path, st))\n@@ -905,7 +905,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(&buf, sha1);\n+\tsha1_file_name(the_repository, &buf, sha1);\n \t*path = buf.buf;\n \n \tfd = git_open(*path);\n@@ -1591,7 +1591,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr,\n \tstatic struct strbuf filename = STRBUF_INIT;\n \n \tstrbuf_reset(&filename);\n-\tsha1_file_name(&filename, oid->hash);\n+\tsha1_file_name(the_repository, &filename, oid->hash);\n \n \tfd = create_tmpfile(&tmp_file, filename.buf);\n \tif (fd < 0) {\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340126","messageId":"20180224004754.129721-23-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 22/27] sha1_file: allow sha1_file_name to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:49Z","receivedAt":"2018-02-24T00:48:35Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 3 +--\n sha1_file.c | 4 ++--\n 2 files changed, 3 insertions(+), 4 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex dbd589e0083..8b38330d256 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -117,8 +117,7 @@ void raw_object_store_clear(struct raw_object_store *o);\n * Put in `buf` the name of the file in the local object database that\n * would be used to store a loose object with the specified sha1.\n */\n-#define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)\n-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);\n+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);\n \n #define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)\n void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 831e2da3572..59993a0878d 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -321,9 +321,9 @@ static void fill_sha1_path(struct strbuf *buf, const unsigned char *sha1)\n \t}\n }\n \n-void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1)\n+void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1)\n {\n-\tstrbuf_addstr(buf, get_object_directory());\n+\tstrbuf_addstr(buf, r->objects.objectdir);\n \tstrbuf_addch(buf, '/');\n \tfill_sha1_path(buf, sha1);\n }\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340127","messageId":"20180224004754.129721-24-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 23/27] sha1_file: allow stat_sha1_file to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:50Z","receivedAt":"2018-02-24T00:48:37Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 +++++------\n 1 file changed, 5 insertions(+), 6 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 59993a0878d..952deb02195 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -869,23 +869,22 @@ int git_open_cloexec(const char *name, int flags)\n * Note that it may point to static storage and is only valid until another\n * call to sha1_file_name(), etc.\n */\n-#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)\n-static int stat_sha1_file_the_repository(const unsigned char *sha1,\n-\t\t\t\t\t struct stat *st, const char **path)\n+static int stat_sha1_file(struct repository *r, const unsigned char *sha1,\n+\t\t\t struct stat *st, const char **path)\n {\n \tstruct alternate_object_database *alt;\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(the_repository, &buf, sha1);\n+\tsha1_file_name(r, &buf, sha1);\n \t*path = buf.buf;\n \n \tif (!lstat(*path, st))\n \t\treturn 0;\n \n-\tprepare_alt_odb(the_repository);\n+\tprepare_alt_odb(r);\n \terrno = ENOENT;\n-\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = r->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tif (!lstat(*path, st))\n \t\t\treturn 0;\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340128","messageId":"20180224004754.129721-21-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 20/27] sha1_file: add repository argument to map_sha1_file","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:47Z","receivedAt":"2018-02-24T00:48:40Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow map_sha1_file callers to be more\nspecific about which repository to handle. This is a small mechanical\nchange; it doesn't change the implementation to handle repositories\nother than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nWhile at it, move the declaration to object-store.h, where it should\nbe easier to find.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n cache.h | 1 -\n object-store.h | 3 +++\n sha1_file.c | 4 ++--\n streaming.c | 5 ++++-\n 4 files changed, 9 insertions(+), 4 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 94d85a5c692..57173991839 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -1226,7 +1226,6 @@ extern int force_object_loose(const struct object_id *oid, time_t mtime);\n \n extern int git_open_cloexec(const char *name, int flags);\n #define git_open(name) git_open_cloexec(name, O_RDONLY)\n-extern void *map_sha1_file(const unsigned char *sha1, unsigned long *size);\n extern int unpack_sha1_header(git_zstream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz);\n extern int parse_sha1_header(const char *hdr, unsigned long *sizep);\n \ndiff --git a/object-store.h b/object-store.h\nindex c0d1292cfc2..dbd589e0083 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -120,4 +120,7 @@ void raw_object_store_clear(struct raw_object_store *o);\n #define sha1_file_name(r, b, s) sha1_file_name_##r(b, s)\n void sha1_file_name_the_repository(struct strbuf *buf, const unsigned char *sha1);\n \n+#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)\n+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);\n+\n #endif /* OBJECT_STORE_H */\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d4af8f56814..6ec43e745d7 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -963,7 +963,7 @@ static void *map_sha1_file_1_the_repository(const char *path,\n \treturn map;\n }\n \n-void *map_sha1_file(const unsigned char *sha1, unsigned long *size)\n+void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)\n {\n \treturn map_sha1_file_1(the_repository, NULL, sha1, size);\n }\n@@ -1187,7 +1187,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,\n \t\treturn 0;\n \t}\n \n-\tmap = map_sha1_file(sha1, &mapsize);\n+\tmap = map_sha1_file(the_repository, sha1, &mapsize);\n \tif (!map)\n \t\treturn -1;\n \ndiff --git a/streaming.c b/streaming.c\nindex 5892b50bd89..22d27df55eb 100644\n--- a/streaming.c\n+++ b/streaming.c\n@@ -3,6 +3,8 @@\n */\n #include \"cache.h\"\n #include \"streaming.h\"\n+#include \"repository.h\"\n+#include \"object-store.h\"\n #include \"packfile.h\"\n \n enum input_source {\n@@ -335,7 +337,8 @@ static struct stream_vtbl loose_vtbl = {\n \n static open_method_decl(loose)\n {\n-\tst->u.loose.mapped = map_sha1_file(sha1, &st->u.loose.mapsize);\n+\tst->u.loose.mapped = map_sha1_file(the_repository,\n+\t\t\t\t\t sha1, &st->u.loose.mapsize);\n \tif (!st->u.loose.mapped)\n \t\treturn -1;\n \tif ((unpack_sha1_header(&st->z,\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340129","messageId":"20180224004754.129721-27-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 26/27] sha1_file: allow map_sha1_file to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:53Z","receivedAt":"2018-02-24T00:48:43Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 3 +--\n sha1_file.c | 5 +++--\n 2 files changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 8b38330d256..afe2f93459b 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -119,7 +119,6 @@ void raw_object_store_clear(struct raw_object_store *o);\n */\n void sha1_file_name(struct repository *r, struct strbuf *buf, const unsigned char *sha1);\n \n-#define map_sha1_file(r, s, sz) map_sha1_file_##r(s, sz)\n-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size);\n+void *map_sha1_file(struct repository *r, const unsigned char *sha1, unsigned long *size);\n \n #endif /* OBJECT_STORE_H */\ndiff --git a/sha1_file.c b/sha1_file.c\nindex e32f1da6b69..902cba42106 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -959,9 +959,10 @@ static void *map_sha1_file_1(struct repository *r, const char *path,\n \treturn map;\n }\n \n-void *map_sha1_file_the_repository(const unsigned char *sha1, unsigned long *size)\n+void *map_sha1_file(struct repository *r,\n+\t\t const unsigned char *sha1, unsigned long *size)\n {\n-\treturn map_sha1_file_1(the_repository, NULL, sha1, size);\n+\treturn map_sha1_file_1(r, NULL, sha1, size);\n }\n \n static int unpack_sha1_short_header(git_zstream *stream,\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340130","messageId":"20180224004754.129721-28-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 27/27] sha1_file: allow sha1_loose_object_info to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:54Z","receivedAt":"2018-02-24T00:48:45Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Jonathan Nieder \n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 +++++------\n 1 file changed, 5 insertions(+), 6 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 902cba42106..dfc8deec38e 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -1151,10 +1151,9 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)\n \treturn parse_sha1_header_extended(hdr, &oi, 0);\n }\n \n-#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)\n-static int sha1_loose_object_info_the_repository(const unsigned char *sha1,\n-\t\t\t\t\t\t struct object_info *oi,\n-\t\t\t\t\t\t int flags)\n+static int sha1_loose_object_info(struct repository *r,\n+\t\t\t\t const unsigned char *sha1,\n+\t\t\t\t struct object_info *oi, int flags)\n {\n \tint status = 0;\n \tunsigned long mapsize;\n@@ -1178,14 +1177,14 @@ static int sha1_loose_object_info_the_repository(const unsigned char *sha1,\n \tif (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {\n \t\tconst char *path;\n \t\tstruct stat st;\n-\t\tif (stat_sha1_file(the_repository, sha1, &st, &path) < 0)\n+\t\tif (stat_sha1_file(r, sha1, &st, &path) < 0)\n \t\t\treturn -1;\n \t\tif (oi->disk_sizep)\n \t\t\t*oi->disk_sizep = st.st_size;\n \t\treturn 0;\n \t}\n \n-\tmap = map_sha1_file(the_repository, sha1, &mapsize);\n+\tmap = map_sha1_file(r, sha1, &mapsize);\n \tif (!map)\n \t\treturn -1;\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340131","messageId":"20180224004754.129721-22-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 21/27] sha1_file: add repository argument to sha1_loose_object_info","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:48Z","receivedAt":"2018-02-24T00:48:48Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the sha1_loose_object_info caller\nto be more specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 6ec43e745d7..831e2da3572 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -1154,9 +1154,10 @@ int parse_sha1_header(const char *hdr, unsigned long *sizep)\n \treturn parse_sha1_header_extended(hdr, &oi, 0);\n }\n \n-static int sha1_loose_object_info(const unsigned char *sha1,\n-\t\t\t\t struct object_info *oi,\n-\t\t\t\t int flags)\n+#define sha1_loose_object_info(r, s, o, f) sha1_loose_object_info_##r(s, o, f)\n+static int sha1_loose_object_info_the_repository(const unsigned char *sha1,\n+\t\t\t\t\t\t struct object_info *oi,\n+\t\t\t\t\t\t int flags)\n {\n \tint status = 0;\n \tunsigned long mapsize;\n@@ -1275,7 +1276,7 @@ int sha1_object_info_extended(const unsigned char *sha1, struct object_info *oi,\n \t\t\tbreak;\n \n \t\t/* Most likely it's a loose object. */\n-\t\tif (!sha1_loose_object_info(real, oi, flags))\n+\t\tif (!sha1_loose_object_info(the_repository, real, oi, flags))\n \t\t\treturn 0;\n \n \t\t/* Not a loose object; someone else may have just packed it. */\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340132","messageId":"20180224004754.129721-26-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 25/27] sha1_file: allow map_sha1_file_1 to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:52Z","receivedAt":"2018-02-24T00:48:49Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"From: Jonathan Nieder \n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 8 +++-----\n 1 file changed, 3 insertions(+), 5 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 73bbe3a3e88..e32f1da6b69 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -931,10 +931,8 @@ static int open_sha1_file(struct repository *r,\n * Map the loose object at \"path\" if it is not NULL, or the path found by\n * searching for a loose object named \"sha1\".\n */\n-#define map_sha1_file_1(r, p, s, si) map_sha1_file_1_##r(p, s, si)\n-static void *map_sha1_file_1_the_repository(const char *path,\n-\t\t\t\t\t const unsigned char *sha1,\n-\t\t\t\t\t unsigned long *size)\n+static void *map_sha1_file_1(struct repository *r, const char *path,\n+\t\t\t const unsigned char *sha1, unsigned long *size)\n {\n \tvoid *map;\n \tint fd;\n@@ -942,7 +940,7 @@ static void *map_sha1_file_1_the_repository(const char *path,\n \tif (path)\n \t\tfd = git_open(path);\n \telse\n-\t\tfd = open_sha1_file(the_repository, sha1, &path);\n+\t\tfd = open_sha1_file(r, sha1, &path);\n \tmap = NULL;\n \tif (fd >= 0) {\n \t\tstruct stat st;\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340133","messageId":"20180224004754.129721-11-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 10/27] sha1_file: add repository argument to link_alt_odb_entry","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:37Z","receivedAt":"2018-02-24T00:48:54Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the link_alt_odb_entry caller to be\nmore specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nSince the implementation does not yet work with other repositories,\nuse a wrapper macro to enforce that the caller passes in\nthe_repository as the first argument. It would be more appealing to\nuse BUILD_ASSERT_OR_ZERO to enforce this, but that doesn't work\nbecause it requires a compile-time constant and common compilers like\ngcc 4.8.4 do not consider \"r == the_repository\" a compile-time\nconstant.\n\nThis and the following three patches add repository arguments to\nlink_alt_odb_entry, read_info_alternates, link_alt_odb_entries\nand prepare_alt_odb. Three out of the four functions are found\nin a recursive call chain, calling each other, and one of them\naccesses the repositories `objectdir` (which was migrated; it\nwas an obvious choice) and `ignore_env` (which we need to keep in\nthe repository struct for clarify); hence we will pass through the\nrepository unlike just the object store object + the ignore_env flag.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 8e6f213f9d9..e5fe0aa04ef 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -388,8 +388,9 @@ static int alt_odb_usable(struct raw_object_store *o,\n * terminating NUL.\n */\n static void read_info_alternates(const char * relative_base, int depth);\n-static int link_alt_odb_entry(const char *entry, const char *relative_base,\n-\tint depth, const char *normalized_objdir)\n+#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)\n+static int link_alt_odb_entry_the_repository(const char *entry,\n+\tconst char *relative_base, int depth, const char *normalized_objdir)\n {\n \tstruct alternate_object_database *ent;\n \tstruct strbuf pathbuf = STRBUF_INIT;\n@@ -486,7 +487,8 @@ static void link_alt_odb_entries(const char *alt, int sep,\n \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n \t\tif (!entry.len)\n \t\t\tcontinue;\n-\t\tlink_alt_odb_entry(entry.buf, relative_base, depth, objdirbuf.buf);\n+\t\tlink_alt_odb_entry(the_repository, entry.buf,\n+\t\t\t\t relative_base, depth, objdirbuf.buf);\n \t}\n \tstrbuf_release(&entry);\n \tstrbuf_release(&objdirbuf);\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340134","messageId":"20180224004754.129721-25-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 24/27] sha1_file: allow open_sha1_file to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:51Z","receivedAt":"2018-02-24T00:48:56Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Signed-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 +++++------\n 1 file changed, 5 insertions(+), 6 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 952deb02195..73bbe3a3e88 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -897,9 +897,8 @@ static int stat_sha1_file(struct repository *r, const unsigned char *sha1,\n * Like stat_sha1_file(), but actually open the object and return the\n * descriptor. See the caveats on the \"path\" parameter above.\n */\n-#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)\n-static int open_sha1_file_the_repository(const unsigned char *sha1,\n-\t\t\t\t\t const char **path)\n+static int open_sha1_file(struct repository *r,\n+\t\t\t const unsigned char *sha1, const char **path)\n {\n \tint fd;\n \tstruct alternate_object_database *alt;\n@@ -907,7 +906,7 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,\n \tstatic struct strbuf buf = STRBUF_INIT;\n \n \tstrbuf_reset(&buf);\n-\tsha1_file_name(the_repository, &buf, sha1);\n+\tsha1_file_name(r, &buf, sha1);\n \t*path = buf.buf;\n \n \tfd = git_open(*path);\n@@ -915,8 +914,8 @@ static int open_sha1_file_the_repository(const unsigned char *sha1,\n \t\treturn fd;\n \tmost_interesting_errno = errno;\n \n-\tprepare_alt_odb(the_repository);\n-\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n+\tprepare_alt_odb(r);\n+\tfor (alt = r->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tfd = git_open(*path);\n \t\tif (fd >= 0)\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340135","messageId":"20180224004754.129721-18-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 17/27] sha1_file: add repository argument to stat_sha1_file","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:44Z","receivedAt":"2018-02-24T00:48:58Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the stat_sha1_file caller to be\nmore specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 9 +++++----\n 1 file changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 8ef2f856035..b7e6e4a9bfc 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -869,8 +869,9 @@ int git_open_cloexec(const char *name, int flags)\n * Note that it may point to static storage and is only valid until another\n * call to sha1_file_name(), etc.\n */\n-static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n-\t\t\t const char **path)\n+#define stat_sha1_file(r, s, st, p) stat_sha1_file_##r(s, st, p)\n+static int stat_sha1_file_the_repository(const unsigned char *sha1,\n+\t\t\t\t\t struct stat *st, const char **path)\n {\n \tstruct alternate_object_database *alt;\n \tstatic struct strbuf buf = STRBUF_INIT;\n@@ -1176,7 +1177,7 @@ static int sha1_loose_object_info(const unsigned char *sha1,\n \tif (!oi->typep && !oi->typename && !oi->sizep && !oi->contentp) {\n \t\tconst char *path;\n \t\tstruct stat st;\n-\t\tif (stat_sha1_file(sha1, &st, &path) < 0)\n+\t\tif (stat_sha1_file(the_repository, sha1, &st, &path) < 0)\n \t\t\treturn -1;\n \t\tif (oi->disk_sizep)\n \t\t\t*oi->disk_sizep = st.st_size;\n@@ -1390,7 +1391,7 @@ void *read_sha1_file_extended(const unsigned char *sha1,\n \t\tdie(\"replacement %s not found for %s\",\n \t\t sha1_to_hex(repl), sha1_to_hex(sha1));\n \n-\tif (!stat_sha1_file(repl, &st, &path))\n+\tif (!stat_sha1_file(the_repository, repl, &st, &path))\n \t\tdie(\"loose object %s (stored in %s) is corrupt\",\n \t\t sha1_to_hex(repl), path);\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340136","messageId":"20180224004754.129721-19-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 18/27] sha1_file: add repository argument to open_sha1_file","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:45Z","receivedAt":"2018-02-24T00:49:01Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a repository argument to allow the open_sha1_file caller to be\nmore specific about which repository to act on. This is a small\nmechanical change; it doesn't change the implementation to handle\nrepositories other than the_repository yet.\n\nAs with the previous commits, use a macro to catch callers passing a\nrepository other than the_repository at compile time.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 6 ++++--\n 1 file changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex b7e6e4a9bfc..7e471e0cf4b 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -898,7 +898,9 @@ static int stat_sha1_file_the_repository(const unsigned char *sha1,\n * Like stat_sha1_file(), but actually open the object and return the\n * descriptor. See the caveats on the \"path\" parameter above.\n */\n-static int open_sha1_file(const unsigned char *sha1, const char **path)\n+#define open_sha1_file(r, s, p) open_sha1_file_##r(s, p)\n+static int open_sha1_file_the_repository(const unsigned char *sha1,\n+\t\t\t\t\t const char **path)\n {\n \tint fd;\n \tstruct alternate_object_database *alt;\n@@ -941,7 +943,7 @@ static void *map_sha1_file_1(const char *path,\n \tif (path)\n \t\tfd = git_open(path);\n \telse\n-\t\tfd = open_sha1_file(sha1, &path);\n+\t\tfd = open_sha1_file(the_repository, sha1, &path);\n \tmap = NULL;\n \tif (fd >= 0) {\n \t\tstruct stat st;\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340137","messageId":"20180224004754.129721-14-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 13/27] sha1_file: add repository argument to prepare_alt_odb","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:40Z","receivedAt":"2018-02-24T00:49:07Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"See previous patch for explanation.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n builtin/fsck.c | 2 +-\n object-store.h | 3 ++-\n packfile.c | 2 +-\n sha1_file.c | 13 +++++++------\n sha1_name.c | 2 +-\n 5 files changed, 12 insertions(+), 10 deletions(-)\n\ndiff --git a/builtin/fsck.c b/builtin/fsck.c\nindex c3a339193c8..cdcf2fa926c 100644\n--- a/builtin/fsck.c\n+++ b/builtin/fsck.c\n@@ -722,7 +722,7 @@ int cmd_fsck(int argc, const char **argv, const char *prefix)\n \t} else {\n \t\tfsck_object_dir(get_object_directory());\n \n-\t\tprepare_alt_odb();\n+\t\tprepare_alt_odb(the_repository);\n \t\tfor (alt = the_repository->objects.alt_odb_list;\n \t\t alt; alt = alt->next)\n \t\t\tfsck_object_dir(alt->path);\ndiff --git a/object-store.h b/object-store.h\nindex 31ca4d3f340..95f328482aa 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -20,7 +20,8 @@ struct alternate_object_database {\n \n \tchar path[FLEX_ARRAY];\n };\n-void prepare_alt_odb(void);\n+#define prepare_alt_odb(r) prepare_alt_odb_##r()\n+void prepare_alt_odb_the_repository(void);\n char *compute_alternate_path(const char *path, struct strbuf *err);\n typedef int alt_odb_fn(struct alternate_object_database *, void *);\n int foreach_alt_odb(alt_odb_fn, void*);\ndiff --git a/packfile.c b/packfile.c\nindex 8d7b1dd5f9c..1e02fe3b0bb 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -889,7 +889,7 @@ void prepare_packed_git(void)\n \tif (the_repository->objects.packed_git_initialized)\n \t\treturn;\n \tprepare_packed_git_one(get_object_directory(), 1);\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next)\n \t\tprepare_packed_git_one(alt->path, 0);\n \trearrange_packed_git();\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 8fb56ca0362..381eaf8221d 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -23,6 +23,7 @@\n #include \"sha1-lookup.h\"\n #include \"bulk-checkin.h\"\n #include \"repository.h\"\n+#include \"object-store.h\"\n #include \"streaming.h\"\n #include \"dir.h\"\n #include \"list.h\"\n@@ -579,7 +580,7 @@ void add_to_alternates_memory(const char *reference)\n \t * Make sure alternates are initialized, or else our entry may be\n \t * overwritten when they are.\n \t */\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \n \tlink_alt_odb_entries(the_repository, reference,\n \t\t\t '\\n', NULL, 0);\n@@ -665,7 +666,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \tstruct alternate_object_database *ent;\n \tint r = 0;\n \n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \tfor (ent = the_repository->objects.alt_odb_list; ent; ent = ent->next) {\n \t\tr = fn(ent, cb);\n \t\tif (r)\n@@ -674,7 +675,7 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \treturn r;\n }\n \n-void prepare_alt_odb(void)\n+void prepare_alt_odb_the_repository(void)\n {\n \tconst char *alt;\n \n@@ -728,7 +729,7 @@ static int check_and_freshen_local(const unsigned char *sha1, int freshen)\n static int check_and_freshen_nonlocal(const unsigned char *sha1, int freshen)\n {\n \tstruct alternate_object_database *alt;\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\tconst char *path = alt_sha1_path(alt, sha1);\n \t\tif (check_and_freshen_file(path, freshen))\n@@ -887,7 +888,7 @@ static int stat_sha1_file(const unsigned char *sha1, struct stat *st,\n \tif (!lstat(*path, st))\n \t\treturn 0;\n \n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \terrno = ENOENT;\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n@@ -918,7 +919,7 @@ static int open_sha1_file(const unsigned char *sha1, const char **path)\n \t\treturn fd;\n \tmost_interesting_errno = errno;\n \n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n \t\t*path = alt_sha1_path(alt, sha1);\n \t\tfd = git_open(*path);\ndiff --git a/sha1_name.c b/sha1_name.c\nindex 016c883b5c7..627e93f1447 100644\n--- a/sha1_name.c\n+++ b/sha1_name.c\n@@ -353,7 +353,7 @@ static int init_object_disambiguation(const char *name, int len,\n \n \tds->len = len;\n \tds->hex_pfx[len] = '\\0';\n-\tprepare_alt_odb();\n+\tprepare_alt_odb(the_repository);\n \treturn 0;\n }\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340138","messageId":"20180224004754.129721-15-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 14/27] sha1_file: allow link_alt_odb_entries to handle arbitrary repositories","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:41Z","receivedAt":"2018-02-24T00:49:09Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Actually this also allows read_info_alternates and link_alt_odb_entry to\nhandle arbitrary repositories, but link_alt_odb_entries is the most\ninteresting function in this set of functions, hence the commit subject.\n\nThese functions span a strongly connected component in the function\ngraph, i.e. the recursive call chain might look like\n\n -> link_alt_odb_entries\n -> link_alt_odb_entry\n -> read_info_alternates\n -> link_alt_odb_entries\n\nThat is why we need to convert all these functions at the same time.\n\nSigned-off-by: Jonathan Nieder \nSigned-off-by: Stefan Beller \n---\n object-store.h | 4 ++++\n sha1_file.c | 36 ++++++++++++++++--------------------\n 2 files changed, 20 insertions(+), 20 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 95f328482aa..08cd48ade11 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -18,6 +18,10 @@ struct alternate_object_database {\n \tchar loose_objects_subdir_seen[256];\n \tstruct oid_array loose_objects_cache;\n \n+\t/*\n+\t * Path to the alternative object store. If this is a relative path,\n+\t * it is relative to the current working directory.\n+\t */\n \tchar path[FLEX_ARRAY];\n };\n #define prepare_alt_odb(r) prepare_alt_odb_##r()\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 381eaf8221d..ab42d430b8a 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -388,11 +388,10 @@ static int alt_odb_usable(struct raw_object_store *o,\n * SHA1, an extra slash for the first level indirection, and the\n * terminating NUL.\n */\n-#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)\n-static void read_info_alternates_the_repository(const char *relative_base,\n-\t\t\t\t\t\tint depth);\n-#define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)\n-static int link_alt_odb_entry_the_repository(const char *entry,\n+static void read_info_alternates(struct repository *r,\n+\t\t\t\t const char *relative_base,\n+\t\t\t\t int depth);\n+static int link_alt_odb_entry(struct repository *r, const char *entry,\n \tconst char *relative_base, int depth, const char *normalized_objdir)\n {\n \tstruct alternate_object_database *ent;\n@@ -418,7 +417,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,\n \twhile (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')\n \t\tstrbuf_setlen(&pathbuf, pathbuf.len - 1);\n \n-\tif (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {\n+\tif (!alt_odb_usable(&r->objects, &pathbuf, normalized_objdir)) {\n \t\tstrbuf_release(&pathbuf);\n \t\treturn -1;\n \t}\n@@ -426,12 +425,12 @@ static int link_alt_odb_entry_the_repository(const char *entry,\n \tent = alloc_alt_odb(pathbuf.buf);\n \n \t/* add the alternate entry */\n-\t*the_repository->objects.alt_odb_tail = ent;\n-\tthe_repository->objects.alt_odb_tail = &(ent->next);\n+\t*r->objects.alt_odb_tail = ent;\n+\tr->objects.alt_odb_tail = &(ent->next);\n \tent->next = NULL;\n \n \t/* recursively add alternates */\n-\tread_info_alternates(the_repository, pathbuf.buf, depth + 1);\n+\tread_info_alternates(r, pathbuf.buf, depth + 1);\n \n \tstrbuf_release(&pathbuf);\n \treturn 0;\n@@ -466,12 +465,8 @@ static const char *parse_alt_odb_entry(const char *string,\n \treturn end;\n }\n \n-#define link_alt_odb_entries(r, a, s, rb, d) \\\n-\tlink_alt_odb_entries_##r(a, s, rb, d)\n-static void link_alt_odb_entries_the_repository(const char *alt,\n-\t\t\t\t\t\tint sep,\n-\t\t\t\t\t\tconst char *relative_base,\n-\t\t\t\t\t\tint depth)\n+static void link_alt_odb_entries(struct repository *r, const char *alt,\n+\t\t\t\t int sep, const char *relative_base, int depth)\n {\n \tstruct strbuf objdirbuf = STRBUF_INIT;\n \tstruct strbuf entry = STRBUF_INIT;\n@@ -485,7 +480,7 @@ static void link_alt_odb_entries_the_repository(const char *alt,\n \t\treturn;\n \t}\n \n-\tstrbuf_add_absolute_path(&objdirbuf, get_object_directory());\n+\tstrbuf_add_absolute_path(&objdirbuf, r->objects.objectdir);\n \tif (strbuf_normalize_path(&objdirbuf) < 0)\n \t\tdie(\"unable to normalize object directory: %s\",\n \t\t objdirbuf.buf);\n@@ -494,15 +489,16 @@ static void link_alt_odb_entries_the_repository(const char *alt,\n \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n \t\tif (!entry.len)\n \t\t\tcontinue;\n-\t\tlink_alt_odb_entry(the_repository, entry.buf,\n+\t\tlink_alt_odb_entry(r, entry.buf,\n \t\t\t\t relative_base, depth, objdirbuf.buf);\n \t}\n \tstrbuf_release(&entry);\n \tstrbuf_release(&objdirbuf);\n }\n \n-static void read_info_alternates_the_repository(const char *relative_base,\n-\t\t\t\t\t\tint depth)\n+static void read_info_alternates(struct repository *r,\n+\t\t\t\t const char *relative_base,\n+\t\t\t\t int depth)\n {\n \tchar *path;\n \tstruct strbuf buf = STRBUF_INIT;\n@@ -514,7 +510,7 @@ static void read_info_alternates_the_repository(const char *relative_base,\n \t\treturn;\n \t}\n \n-\tlink_alt_odb_entries(the_repository, buf.buf, '\\n', relative_base, depth);\n+\tlink_alt_odb_entries(r, buf.buf, '\\n', relative_base, depth);\n \tstrbuf_release(&buf);\n \tfree(path);\n }\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340139","messageId":"20180224004754.129721-8-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 07/27] pack: move prepare_packed_git_run_once to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:34Z","receivedAt":"2018-02-24T00:49:12Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Each repository's object store can be initialized independently, so\nthey must not share a run_once variable.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n object-store.h | 8 +++++++-\n packfile.c | 7 +++----\n 2 files changed, 10 insertions(+), 5 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 4f768465a14..bd6441e525f 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -89,9 +89,15 @@ struct raw_object_store {\n \n \tstruct alternate_object_database *alt_odb_list;\n \tstruct alternate_object_database **alt_odb_tail;\n+\n+\t/*\n+\t * Whether packed_git has already been populated with this repository's\n+\t * packs.\n+\t */\n+\tunsigned packed_git_initialized : 1;\n };\n \n-#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL }\n+#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0 }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/packfile.c b/packfile.c\nindex 65d9a4f6c61..3ee349ab1bd 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -883,12 +883,11 @@ static void prepare_packed_git_mru(void)\n \t\tlist_add_tail(&p->mru, &the_repository->objects.packed_git_mru);\n }\n \n-static int prepare_packed_git_run_once = 0;\n void prepare_packed_git(void)\n {\n \tstruct alternate_object_database *alt;\n \n-\tif (prepare_packed_git_run_once)\n+\tif (the_repository->objects.packed_git_initialized)\n \t\treturn;\n \tprepare_packed_git_one(get_object_directory(), 1);\n \tprepare_alt_odb();\n@@ -896,13 +895,13 @@ void prepare_packed_git(void)\n \t\tprepare_packed_git_one(alt->path, 0);\n \trearrange_packed_git();\n \tprepare_packed_git_mru();\n-\tprepare_packed_git_run_once = 1;\n+\tthe_repository->objects.packed_git_initialized = 1;\n }\n \n void reprepare_packed_git(void)\n {\n \tapproximate_object_count_valid = 0;\n-\tprepare_packed_git_run_once = 0;\n+\tthe_repository->objects.packed_git_initialized = 0;\n \tprepare_packed_git();\n }\n \n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340140","messageId":"20180224004754.129721-13-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 12/27] sha1_file: add repository argument to link_alt_odb_entries","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:39Z","receivedAt":"2018-02-24T00:49:14Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"See previous patch for explanation.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 19 +++++++++++++------\n 1 file changed, 13 insertions(+), 6 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex f93d3b95b54..8fb56ca0362 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -465,8 +465,12 @@ static const char *parse_alt_odb_entry(const char *string,\n \treturn end;\n }\n \n-static void link_alt_odb_entries(const char *alt, int sep,\n-\t\t\t\t const char *relative_base, int depth)\n+#define link_alt_odb_entries(r, a, s, rb, d) \\\n+\tlink_alt_odb_entries_##r(a, s, rb, d)\n+static void link_alt_odb_entries_the_repository(const char *alt,\n+\t\t\t\t\t\tint sep,\n+\t\t\t\t\t\tconst char *relative_base,\n+\t\t\t\t\t\tint depth)\n {\n \tstruct strbuf objdirbuf = STRBUF_INIT;\n \tstruct strbuf entry = STRBUF_INIT;\n@@ -509,7 +513,7 @@ static void read_info_alternates_the_repository(const char *relative_base,\n \t\treturn;\n \t}\n \n-\tlink_alt_odb_entries(buf.buf, '\\n', relative_base, depth);\n+\tlink_alt_odb_entries(the_repository, buf.buf, '\\n', relative_base, depth);\n \tstrbuf_release(&buf);\n \tfree(path);\n }\n@@ -563,7 +567,8 @@ void add_to_alternates_file(const char *reference)\n \t\tif (commit_lock_file(&lock))\n \t\t\tdie_errno(\"unable to move new alternates file into place\");\n \t\tif (the_repository->objects.alt_odb_tail)\n-\t\t\tlink_alt_odb_entries(reference, '\\n', NULL, 0);\n+\t\t\tlink_alt_odb_entries(the_repository, reference,\n+\t\t\t\t\t '\\n', NULL, 0);\n \t}\n \tfree(alts);\n }\n@@ -576,7 +581,8 @@ void add_to_alternates_memory(const char *reference)\n \t */\n \tprepare_alt_odb();\n \n-\tlink_alt_odb_entries(reference, '\\n', NULL, 0);\n+\tlink_alt_odb_entries(the_repository, reference,\n+\t\t\t '\\n', NULL, 0);\n }\n \n /*\n@@ -679,7 +685,8 @@ void prepare_alt_odb(void)\n \n \tthe_repository->objects.alt_odb_tail =\n \t\t\t&the_repository->objects.alt_odb_list;\n-\tlink_alt_odb_entries(alt, PATH_SEP, NULL, 0);\n+\tlink_alt_odb_entries(the_repository, alt,\n+\t\t\t PATH_SEP, NULL, 0);\n \n \tread_info_alternates(the_repository, get_object_directory(), 0);\n }\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340141","messageId":"20180224004754.129721-10-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 09/27] sha1_file: add raw_object_store argument to alt_odb_usable","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:36Z","receivedAt":"2018-02-24T00:49:18Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Add a raw_object_store to alt_odb_usable to be more specific about which\nrepository to act on. The choice of the repository is delegated to its\nonly caller link_alt_odb_entry.\n\nSigned-off-by: Stefan Beller \n---\n sha1_file.c | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex ac25146076e..8e6f213f9d9 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -344,7 +344,9 @@ static const char *alt_sha1_path(struct alternate_object_database *alt,\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\n-static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)\n+static int alt_odb_usable(struct raw_object_store *o,\n+\t\t\t struct strbuf *path,\n+\t\t\t const char *normalized_objdir)\n {\n \tstruct alternate_object_database *alt;\n \n@@ -360,7 +362,7 @@ static int alt_odb_usable(struct strbuf *path, const char *normalized_objdir)\n \t * Prevent the common mistake of listing the same\n \t * thing twice, or object directory itself.\n \t */\n-\tfor (alt = the_repository->objects.alt_odb_list; alt; alt = alt->next) {\n+\tfor (alt = o->alt_odb_list; alt; alt = alt->next) {\n \t\tif (!fspathcmp(path->buf, alt->path))\n \t\t\treturn 0;\n \t}\n@@ -412,7 +414,7 @@ static int link_alt_odb_entry(const char *entry, const char *relative_base,\n \twhile (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')\n \t\tstrbuf_setlen(&pathbuf, pathbuf.len - 1);\n \n-\tif (!alt_odb_usable(&pathbuf, normalized_objdir)) {\n+\tif (!alt_odb_usable(&the_repository->objects, &pathbuf, normalized_objdir)) {\n \t\tstrbuf_release(&pathbuf);\n \t\treturn -1;\n \t}\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340142","messageId":"20180224004754.129721-12-sbeller@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"[PATCHv4 11/27] sha1_file: add repository argument to read_info_alternates","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-24T00:47:38Z","receivedAt":"2018-02-24T00:49:20Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"See previous patch for explanation.\n\nSigned-off-by: Stefan Beller \nSigned-off-by: Jonathan Nieder \n---\n sha1_file.c | 11 +++++++----\n 1 file changed, 7 insertions(+), 4 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex e5fe0aa04ef..f93d3b95b54 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -387,7 +387,9 @@ static int alt_odb_usable(struct raw_object_store *o,\n * SHA1, an extra slash for the first level indirection, and the\n * terminating NUL.\n */\n-static void read_info_alternates(const char * relative_base, int depth);\n+#define read_info_alternates(r, rb, d) read_info_alternates_##r(rb, d)\n+static void read_info_alternates_the_repository(const char *relative_base,\n+\t\t\t\t\t\tint depth);\n #define link_alt_odb_entry(r, e, rb, d, n) link_alt_odb_entry_##r(e, rb, d, n)\n static int link_alt_odb_entry_the_repository(const char *entry,\n \tconst char *relative_base, int depth, const char *normalized_objdir)\n@@ -428,7 +430,7 @@ static int link_alt_odb_entry_the_repository(const char *entry,\n \tent->next = NULL;\n \n \t/* recursively add alternates */\n-\tread_info_alternates(pathbuf.buf, depth + 1);\n+\tread_info_alternates(the_repository, pathbuf.buf, depth + 1);\n \n \tstrbuf_release(&pathbuf);\n \treturn 0;\n@@ -494,7 +496,8 @@ static void link_alt_odb_entries(const char *alt, int sep,\n \tstrbuf_release(&objdirbuf);\n }\n \n-static void read_info_alternates(const char * relative_base, int depth)\n+static void read_info_alternates_the_repository(const char *relative_base,\n+\t\t\t\t\t\tint depth)\n {\n \tchar *path;\n \tstruct strbuf buf = STRBUF_INIT;\n@@ -678,7 +681,7 @@ void prepare_alt_odb(void)\n \t\t\t&the_repository->objects.alt_odb_list;\n \tlink_alt_odb_entries(alt, PATH_SEP, NULL, 0);\n \n-\tread_info_alternates(get_object_directory(), 0);\n+\tread_info_alternates(the_repository, get_object_directory(), 0);\n }\n \n /* Returns 1 if we have successfully freshened the file, 0 otherwise. */\n-- \n2.16.1.291.g4437f3f132-goog\n\n"},{"id":"340148","messageId":"CACsJy8CHaFEpmaFW0bSQrkWd5V4nPRxQqN+6MRD_86xoibDxtQ@mail.gmail.com","threadId":"47823","inReplyTo":"20180223221746.GJ6619@genre.crustytoothpaste.net","subject":"Re: [PATCH 1/2] setup.c: initialize the_repository correctly in all cases","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-24T03:17:52Z","receivedAt":"2018-02-24T03:18:28Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sat, Feb 24, 2018 at 5:17 AM, brian m. carlson\n wrote:\n> On Fri, Feb 23, 2018 at 04:56:39PM +0700, Nguyễn Thái Ngọc Duy wrote:\n>> [3] the reason server side is still running ok with no hash algo\n>> before [2] is because the programs that use enter_repo() do very\n>> little then spawn a new program (like pack-objects or\n>> upload-archive) to do the heavy lifting. These programs already\n>> use setup_git_dir..()\n>\n> You have \"..()\" here. Did you want to say \"().\" instead?\n\nIt's because we have two functions, setup_git_directory() and\nsetup_git_directory_gently() and I tend to just put '*' instead of\nlisting both. But since it causes confusion, I'm listing both in v2.\n-- \nDuy\n"},{"id":"340149","messageId":"20180224033429.9656-1-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180223095640.25876-1-pclouds@gmail.com","subject":"[PATCH v2 0/5] Fix initializing the_hash_algo","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-24T03:34:24Z","receivedAt":"2018-02-24T03:34:47Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Brian questioned the unnecessary alarms in v1 when \"git diff\" or \"git\nindex-pack\" run in no-repository mode. I had the same feeling after\nsending v1. But instead of suppresing alarms, we could do better.\n\nv2 breaks those \"fall back to SHA-1\" code into separate patches and\nhandles it properly (I hope) instead of blind fall back like v1:\n\n- for index-pack, we can determine the needed algorithm from the pack\n file. I'm making an assumption here that pack files with new hash\n algo must step up file format version. But I think it's a reasonable\n assumption.\n\n- for diff --no-index, I still fall back to SHA-1 to generate the\n hashes like before. We could probably introduce a new command line\n option to use a different hash. But that work could be done later\n when an actual new hash has come\n\nNote, I didn't test but this series could potentially break 'pu' (in a\ngood way). I initially was puzzled why the test suite didn't fail when\nthe_repository->hash_algo was NULL (i.e. before Brian's fix). Then I\nfound out that more work to abstract away SHA-1 hashing functions have\nbeen done since then. But because the_hash_algo is pre-initialized\nwith SHA-1, these special cases did not show up until now. If there\nare more the_hash_algo conversion on 'pu', more \"no-repo\" cases\ncould be spotted by the test suite.\n\nI think this makes pre-initializing the_hash_algo to NULL a very good\npoint: during the abstraction work, we at least must identify the use\ncases like this (code running without repo). We don't have to fix it\nright away, but we can start thinking about how to deal with it. If we\nignore it and switch to a new hash, these code will keep using SHA-1\nand cause more headache in future.\n\nNguyễn Thái Ngọc Duy (5):\n setup.c: initialize the_repository correctly in all cases\n sha1_file.c: keep a good name for \"unknown\" hash_algos[UNKNOWN]\n index-pack: check (and optionally set) hash algo based on input file\n diff.c: initialize hash algo when running in --no-index mode\n Revert \"repository: pre-initialize hash algo pointer\"\n\n builtin/index-pack.c | 26 +++++++++++++++++++++++++-\n builtin/init-db.c | 3 ++-\n cache.h | 3 ++-\n common-main.c | 10 ++++++++++\n diff.c | 12 ++++++++++++\n path.c | 2 +-\n repository.c | 2 +-\n setup.c | 5 ++++-\n sha1_file.c | 2 +-\n t/helper/test-dump-split-index.c | 2 ++\n 10 files changed, 60 insertions(+), 7 deletions(-)\n\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340150","messageId":"20180224033429.9656-2-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180224033429.9656-1-pclouds@gmail.com","subject":"[PATCH v2 1/5] setup.c: initialize the_repository correctly in all cases","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-24T03:34:25Z","receivedAt":"2018-02-24T03:34:51Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"There are many ways for any command to access a git repository:\n\n- most of them will try to discover the .git dir via\n setup_git_directory() and friends\n\n- the server side programs already know where the repo is and prepare\n with enter_repo()\n\n- special commands that deal with repo creation (init/clone) use\n init_db() once the new repo is ready for access.\n\n- somebody accesses $GIT_DIR before any of above functions are called\n and accidentally sets up a git repository by set_git_dir() alone\n\n\"the_repository\" is partially set up via set_git_dir() at some point\nin all four cases. The hash algorithm though is configured later after\n.git/config is read.\n\nSo far proper repo initialization is done only for the first case [1].\nThe second case is not covered (but that's fine [3]). The third case\nwas found and worked around in [2]. The fourth case is a buggy one,\nwhich should be fixed already by jk/no-looking-at-dotgit-outside-repo\nand never happens again.\n\nThis patch makes sure all cases initialize the hash algorithm in\nthe_repository correctly. Both second and third cases must run\ncheck_repo_format() before \"entering\" it. Eventually we probably just\nrename this function to init_repo() or something.\n\n[1] 78a6766802 (Integrate hash algorithm support with repo setup -\n 2017-11-12)\n\n[2] e26f7f19b6 (repository: pre-initialize hash algo pointer -\n 2018-01-19)\n\n[3] the reason server side is still running ok with no hash algo before\n [2] is because the programs that use enter_repo() do very\n little (and unlikely to hash anything) then spawn a new\n program (like pack-objects or upload-archive) to do the heavy\n lifting. These programs already use setup_git_directory() or the\n gently version\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/init-db.c | 3 ++-\n cache.h | 3 ++-\n path.c | 2 +-\n setup.c | 5 ++++-\n 4 files changed, 9 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex c9b7946bad..d119d9906b 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -9,6 +9,7 @@\n #include \"builtin.h\"\n #include \"exec_cmd.h\"\n #include \"parse-options.h\"\n+#include \"repository.h\"\n \n #ifndef DEFAULT_GIT_TEMPLATE_DIR\n #define DEFAULT_GIT_TEMPLATE_DIR \"/usr/share/git-core/templates\"\n@@ -369,7 +370,7 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \t * config file, so this will not fail. What we are catching\n \t * is an attempt to reinitialize new repository with an old tool.\n \t */\n-\tcheck_repository_format();\n+\tcheck_repository_format(the_repository);\n \n \treinit = create_default_files(template_dir, original_git_dir);\n \ndiff --git a/cache.h b/cache.h\nindex 21fbcc2414..6b97138264 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -894,6 +894,7 @@ extern int repository_format_precious_objects;\n extern char *repository_format_partial_clone;\n extern const char *core_partial_clone_filter_default;\n \n+struct repository;\n struct repository_format {\n \tint version;\n \tint precious_objects;\n@@ -926,7 +927,7 @@ int verify_repository_format(const struct repository_format *format,\n * set_git_dir() before calling this, and use it only for \"are we in a valid\n * repo?\".\n */\n-extern void check_repository_format(void);\n+extern void check_repository_format(struct repository *);\n \n #define MTIME_CHANGED\t0x0001\n #define CTIME_CHANGED\t0x0002\ndiff --git a/path.c b/path.c\nindex da8b655730..a544252198 100644\n--- a/path.c\n+++ b/path.c\n@@ -827,7 +827,7 @@ const char *enter_repo(const char *path, int strict)\n \n \tif (is_git_directory(\".\")) {\n \t\tset_git_dir(\".\");\n-\t\tcheck_repository_format();\n+\t\tcheck_repository_format(the_repository);\n \t\treturn path;\n \t}\n \ndiff --git a/setup.c b/setup.c\nindex c5d55dcee4..a82103832e 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -1180,11 +1180,14 @@ int git_config_perm(const char *var, const char *value)\n \treturn -(i & 0666);\n }\n \n-void check_repository_format(void)\n+/* optionally configure \"repo\" to the correct format */\n+void check_repository_format(struct repository *repo)\n {\n \tstruct repository_format repo_fmt;\n \tcheck_repository_format_gently(get_git_dir(), &repo_fmt, NULL);\n \tstartup_info->have_repository = 1;\n+\tif (repo)\n+\t\trepo_set_hash_algo(repo, repo_fmt.hash_algo);\n }\n \n /*\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340151","messageId":"20180224033429.9656-3-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180224033429.9656-1-pclouds@gmail.com","subject":"[PATCH v2 2/5] sha1_file.c: keep a good name for \"unknown\" hash_algos[UNKNOWN]","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-24T03:34:26Z","receivedAt":"2018-02-24T03:34:56Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This is mostly for displaying the hash algorithm name when we report\nerrors. Printing \"unknown\" with '%s' is much better than '(null)' in\nglibc printf version (and probably could crash if other implementations\ndo not check for NULL pointer)\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n sha1_file.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 826d7a0ae3..4b266b1c68 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -71,7 +71,7 @@ static void git_hash_unknown_final(unsigned char *hash, git_hash_ctx *ctx)\n \n const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = {\n \t{\n-\t\tNULL,\n+\t\t\"unknown\",\n \t\t0x00000000,\n \t\t0,\n \t\t0,\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340152","messageId":"20180224033429.9656-4-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180224033429.9656-1-pclouds@gmail.com","subject":"[PATCH v2 3/5] index-pack: check (and optionally set) hash algo based on input file","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-24T03:34:27Z","receivedAt":"2018-02-24T03:35:02Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"After 454253f059 (builtin/index-pack: improve hash function abstraction\n- 2018-02-01), index-pack uses the_hash_algo for hashing. If \"git\nindex-pack\" is executed without a repository, we do not know what hash\nalgorithm to be used and the_hash_algo in theory could be undefined.\n\nSince there should be some information about the hash algorithm in the\ninput pack file, we can initialize the correct hash algorithm with that\nif the_hash_algo is not yet initialized. This assumes that pack files\nwith new hash algorithm MUST step up pack version.\n\nWhile at there, make sure the hash algorithm requested by the pack file\nand configured by the repository (if we're running with a repo) are\nconsistent.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/index-pack.c | 26 +++++++++++++++++++++++++-\n 1 file changed, 25 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 7e3e1a461c..1144458140 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -326,10 +326,31 @@ static const char *open_pack_file(const char *pack_name)\n \t\toutput_fd = -1;\n \t\tnothread_data.pack_fd = input_fd;\n \t}\n-\tthe_hash_algo->init_fn(&input_ctx);\n \treturn pack_name;\n }\n \n+static void prepare_hash_algo(uint32_t pack_version)\n+{\n+\tconst struct git_hash_algo *pack_algo;\n+\n+\tswitch (pack_version) {\n+\tcase 2:\n+\tcase 3:\n+\t\tpack_algo = &hash_algos[GIT_HASH_SHA1];\n+\t\tbreak;\n+\tdefault:\n+\t\tdie(\"BUG: how to determine hash algo for new version?\");\n+\t}\n+\n+\tif (!the_hash_algo)\t/* running without repo */\n+\t\tthe_hash_algo = pack_algo;\n+\n+\tif (the_hash_algo != pack_algo)\n+\t\tdie(_(\"incompatible hash algorithm, \"\n+\t\t \"configured for %s but the pack file needs %s\"),\n+\t\t the_hash_algo->name, pack_algo->name);\n+}\n+\n static void parse_pack_header(void)\n {\n \tstruct pack_header *hdr = fill(sizeof(struct pack_header));\n@@ -341,6 +362,9 @@ static void parse_pack_header(void)\n \t\tdie(_(\"pack version %\"PRIu32\" unsupported\"),\n \t\t\tntohl(hdr->hdr_version));\n \n+\tprepare_hash_algo(ntohl(hdr->hdr_version));\n+\tthe_hash_algo->init_fn(&input_ctx);\n+\n \tnr_objects = ntohl(hdr->hdr_entries);\n \tuse(sizeof(struct pack_header));\n }\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340153","messageId":"20180224033429.9656-5-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180224033429.9656-1-pclouds@gmail.com","subject":"[PATCH v2 4/5] diff.c: initialize hash algo when running in --no-index mode","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-24T03:34:28Z","receivedAt":"2018-02-24T03:35:07Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Our \"git diff\" command supports running as a standalone tool. In this\ncode path, we try to hash the file content but after\n18e2588e11 (sha1_file: switch uses of SHA-1 to the_hash_algo -\n2018-02-01), there is a chance that the_hash_algo (required by\nindex_path) may still be uninitialized if no repository is found.\n\nExecuting index_path() when the_hash_algo is NULL (or points to unknown\nalgo) either crashes or dies. Let's make it a bit safer by explicitly\nfalling back to SHA-1 (so that the diff output remains the same as\nbefore, compared to the alternative that we simply do not hash).\n\ndòng được\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n diff.c | 12 ++++++++++++\n 1 file changed, 12 insertions(+)\n\ndiff --git a/diff.c b/diff.c\nindex 21c3838b25..f5755425b6 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -3995,6 +3995,18 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)\n \t\treturn;\n \t}\n \n+\t/*\n+\t * NEEDSWORK: When running in no-index mode (and no repo is\n+\t * found, thus no hash algo conifugred), fall back to SHA-1\n+\t * hashing (which is used by diff_fill_oid_info below) to\n+\t * avoid regression in diff output.\n+\t *\n+\t * In future, perhaps we can allow the user to specify their\n+\t * hash algorithm from command line in this mode.\n+\t */\n+\tif (o->flags.no_index && !the_hash_algo)\n+\t\tthe_hash_algo = &hash_algos[GIT_HASH_SHA1];\n+\n \tdiff_fill_oid_info(one);\n \tdiff_fill_oid_info(two);\n \n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340154","messageId":"20180224033429.9656-6-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180224033429.9656-1-pclouds@gmail.com","subject":"[PATCH v2 5/5] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-24T03:34:29Z","receivedAt":"2018-02-24T03:35:12Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This reverts commit e26f7f19b6c7485f04234946a59ab8f4fd21d6d1. The root\nproblem, git clone not setting up the_hash_algo, has been fixed in the\nprevious patch.\n\nSince this is a dangerous move and could potentially break stuff after\nrelease (and leads to workaround like the reverted commit), the\nworkaround technically remains, but is hidden behind a new environment\nvariable GIT_HASH_FIXUP. This should let the users continue to use git\nwhile we fix the problem. This variable can be deleted after one or two\nreleases.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n common-main.c | 10 ++++++++++\n repository.c | 2 +-\n t/helper/test-dump-split-index.c | 2 ++\n 3 files changed, 13 insertions(+), 1 deletion(-)\n\ndiff --git a/common-main.c b/common-main.c\nindex 6a689007e7..fbfa98c3b8 100644\n--- a/common-main.c\n+++ b/common-main.c\n@@ -1,6 +1,7 @@\n #include \"cache.h\"\n #include \"exec_cmd.h\"\n #include \"attr.h\"\n+#include \"repository.h\"\n \n /*\n * Many parts of Git have subprograms communicate via pipe, expect the\n@@ -40,5 +41,14 @@ int main(int argc, const char **argv)\n \n \trestore_sigpipe_to_default();\n \n+\t/*\n+\t * Temporary recovery measure while hashing code is being\n+\t * refactored. This variable should be gone after everybody\n+\t * has used the_hash_algo in one or two releases and nobody\n+\t * complains anything.\n+\t */\n+\tif (getenv(\"GIT_HASH_FIXUP\"))\n+\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n+\n \treturn cmd_main(argc, argv);\n }\ndiff --git a/repository.c b/repository.c\nindex 4ffbe9bc94..0d715f4fdb 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -5,7 +5,7 @@\n \n /* The main repository */\n static struct repository the_repo = {\n-\tNULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0\n+\tNULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, NULL, 0, 0\n };\n struct repository *the_repository = &the_repo;\n \ndiff --git a/t/helper/test-dump-split-index.c b/t/helper/test-dump-split-index.c\nindex e44430b699..b759fe3fc1 100644\n--- a/t/helper/test-dump-split-index.c\n+++ b/t/helper/test-dump-split-index.c\n@@ -12,6 +12,8 @@ int cmd_main(int ac, const char **av)\n \tstruct split_index *si;\n \tint i;\n \n+\tsetup_git_directory();\n+\n \tdo_read_index(&the_index, av[1], 1);\n \tprintf(\"own %s\\n\", sha1_to_hex(the_index.sha1));\n \tsi = the_index.split_index;\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340155","messageId":"CACsJy8AUvAv61wM5hnvhyCaOd_oKmMbhAyDGJ9pmkiSu_WveZQ@mail.gmail.com","threadId":"47823","inReplyTo":"20180223222950.GK6619@genre.crustytoothpaste.net","subject":"Re: [PATCH 2/2] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-24T03:44:30Z","receivedAt":"2018-02-24T03:45:09Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sat, Feb 24, 2018 at 5:29 AM, brian m. carlson\n wrote:\n>> @@ -40,5 +41,8 @@ int main(int argc, const char **argv)\n>>\n>> restore_sigpipe_to_default();\n>>\n>> + if (getenv(\"GIT_HASH_FIXUP\"))\n>> + repo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n>\n> I'm lukewarm on adding this environment variable, but considering our\n> history here, we had probably better. We can always remove it after a\n> few releases.\n\nYes that's the intention. But after writing cover letter for v2 and\nsending it out, it looks to me that this thing must stay until all our\ncode is converted to using the_hash_algo (I don't know if there are\nmore to convert or it's finished already). So an alternative is we do\nthe opposite: default to GIT_HASH_SHA1, but when an env variable is\nset, reset it back to NULL. This env variable will _always_ be set by\nthe test suite to help us catch problems.\n-- \nDuy\n"},{"id":"340160","messageId":"CAPig+cQq32eyArKME4_H0njjn2mbPkNVKo=ULbh-16d55tVYOQ@mail.gmail.com","threadId":"47823","inReplyTo":"20180224033429.9656-5-pclouds@gmail.com","subject":"Re: [PATCH v2 4/5] diff.c: initialize hash algo when running in --no-index mode","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-02-24T08:15:41Z","receivedAt":"2018-02-24T08:15:47Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Feb 23, 2018 at 10:34 PM, Nguyễn Thái Ngọc Duy\n wrote:\n> Our \"git diff\" command supports running as a standalone tool. In this\n> code path, we try to hash the file content but after\n> 18e2588e11 (sha1_file: switch uses of SHA-1 to the_hash_algo -\n> 2018-02-01), there is a chance that the_hash_algo (required by\n> index_path) may still be uninitialized if no repository is found.\n>\n> Executing index_path() when the_hash_algo is NULL (or points to unknown\n> algo) either crashes or dies. Let's make it a bit safer by explicitly\n> falling back to SHA-1 (so that the diff output remains the same as\n> before, compared to the alternative that we simply do not hash).\n>\n> dòng được\n\nAccidental paste?\n\n> Signed-off-by: Nguyễn Thái Ngọc Duy \n> ---\n> diff --git a/diff.c b/diff.c\n> @@ -3995,6 +3995,18 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)\n> + /*\n> + * NEEDSWORK: When running in no-index mode (and no repo is\n> + * found, thus no hash algo conifugred), fall back to SHA-1\n\ns/conifugred/configured/\n\n> + * hashing (which is used by diff_fill_oid_info below) to\n> + * avoid regression in diff output.\n> + *\n> + * In future, perhaps we can allow the user to specify their\n> + * hash algorithm from command line in this mode.\n> + */\n> + if (o->flags.no_index && !the_hash_algo)\n> + the_hash_algo = &hash_algos[GIT_HASH_SHA1];\n> +\n> diff_fill_oid_info(one);\n> diff_fill_oid_info(two);\n"},{"id":"340163","messageId":"CACsJy8BjZsSM+g4=hUV+v_Z+Lj2+f8m+9i-OLAR8rz+2Zij0WQ@mail.gmail.com","threadId":"47823","inReplyTo":"CAPig+cQq32eyArKME4_H0njjn2mbPkNVKo=ULbh-16d55tVYOQ@mail.gmail.com","subject":"Re: [PATCH v2 4/5] diff.c: initialize hash algo when running in --no-index mode","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-24T13:45:34Z","receivedAt":"2018-02-24T13:46:09Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sat, Feb 24, 2018 at 3:15 PM, Eric Sunshine wrote:\n> On Fri, Feb 23, 2018 at 10:34 PM, Nguyễn Thái Ngọc Duy\n> wrote:\n>> before, compared to the alternative that we simply do not hash).\n>>\n>> dòng được\n>\n> Accidental paste?\n\nOops. I blame Emacs.\n-- \nDuy\n"},{"id":"340172","messageId":"CACsJy8AUXf4uA-zymrhWUagW84bUzfB6JBQLyaHDJtudTcNy=A@mail.gmail.com","threadId":"47823","inReplyTo":"20180224033429.9656-5-pclouds@gmail.com","subject":"Re: [PATCH v2 4/5] diff.c: initialize hash algo when running in --no-index mode","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-24T14:36:03Z","receivedAt":"2018-02-24T14:36:38Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sat, Feb 24, 2018 at 10:34 AM, Nguyễn Thái Ngọc Duy\n wrote:\n> @@ -3995,6 +3995,18 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)\n> return;\n> }\n>\n> + /*\n> + * NEEDSWORK: When running in no-index mode (and no repo is\n> + * found, thus no hash algo conifugred), fall back to SHA-1\n> + * hashing (which is used by diff_fill_oid_info below) to\n> + * avoid regression in diff output.\n> + *\n> + * In future, perhaps we can allow the user to specify their\n> + * hash algorithm from command line in this mode.\n> + */\n> + if (o->flags.no_index && !the_hash_algo)\n> + the_hash_algo = &hash_algos[GIT_HASH_SHA1];\n\nBrian, are we supposed to use the_hash_algo this way (i.e. as a\nwritable var)? Or should I stick to something like\n\n repo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n\nwhich allows us to notify other parts inside struct repository about\nthe hash algorithm change, if we ever need to?\n\nIf the_hash_algo is supposed to be read-only, maybe I should convert\nthat macro to an inline function to prevent people from accidentally\nreassigning it?\n\n> +\n> diff_fill_oid_info(one);\n> diff_fill_oid_info(two);\n>\n> --\n> 2.16.1.435.g8f24da2e1a\n>\n-- \nDuy\n"},{"id":"340173","messageId":"CACsJy8AS5RiuXeM2u34i0hjKXJqEhsrb4Ysi9Md9OhRpP4_B-A@mail.gmail.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"Re: [PATCHv4 00/27] Moving global state into the repository object (part 1)","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-24T15:00:53Z","receivedAt":"2018-02-24T15:01:29Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"I notice that there are a few global state (or configuration rather)\nleft after this: packed_git_window_size, packed_git_limit and\ndelta_base_cache_limit. These can be changed by $GIT_DIR/config, but\nsince it's still global, a submodule repository will use the same\nsettings of its super repository. If $SUBMODULE/config changes any of\nthese, they are ignored.\n\nThe natural thing to do is move these to raw_object_store too (and\nrepo_submodule_init needs to check submodule's config file). But one\nmay argue these should be per-process instead of per-repo though. I\ndon't know. But I thought I should mention this.\n-- \nDuy\n"},{"id":"340186","messageId":"20180224222900.GA4620@genre.crustytoothpaste.net","threadId":"47823","inReplyTo":"CACsJy8AUXf4uA-zymrhWUagW84bUzfB6JBQLyaHDJtudTcNy=A@mail.gmail.com","subject":"Re: [PATCH v2 4/5] diff.c: initialize hash algo when running in --no-index mode","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2018-02-24T22:29:00Z","receivedAt":"2018-02-24T22:29:14Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Sat, Feb 24, 2018 at 09:36:03PM +0700, Duy Nguyen wrote:\n> On Sat, Feb 24, 2018 at 10:34 AM, Nguyễn Thái Ngọc Duy\n> wrote:\n> > @@ -3995,6 +3995,18 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)\n> > return;\n> > }\n> >\n> > + /*\n> > + * NEEDSWORK: When running in no-index mode (and no repo is\n> > + * found, thus no hash algo conifugred), fall back to SHA-1\n> > + * hashing (which is used by diff_fill_oid_info below) to\n> > + * avoid regression in diff output.\n> > + *\n> > + * In future, perhaps we can allow the user to specify their\n> > + * hash algorithm from command line in this mode.\n> > + */\n> > + if (o->flags.no_index && !the_hash_algo)\n> > + the_hash_algo = &hash_algos[GIT_HASH_SHA1];\n> \n> Brian, are we supposed to use the_hash_algo this way (i.e. as a\n> writable var)? Or should I stick to something like\n> \n> repo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n> \n> which allows us to notify other parts inside struct repository about\n> the hash algorithm change, if we ever need to?\n\nI would definitely recommend using the function. As you pointed out, it\nmakes our code future-proof against needing to more work when setting\nthe value.\n\n> If the_hash_algo is supposed to be read-only, maybe I should convert\n> that macro to an inline function to prevent people from accidentally\n> reassigning it?\n\nYou could if you want to, although I don't really see a need to, since\nwe can just catch it in review. If you wanted to, I'd make it an inline\nfunction for performance reasons.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\nhttps://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"340187","messageId":"20180224223915.GB4620@genre.crustytoothpaste.net","threadId":"47823","inReplyTo":"20180224033429.9656-3-pclouds@gmail.com","subject":"Re: [PATCH v2 2/5] sha1_file.c: keep a good name for \"unknown\" hash_algos[UNKNOWN]","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2018-02-24T22:39:15Z","receivedAt":"2018-02-24T22:40:27Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Sat, Feb 24, 2018 at 10:34:26AM +0700, Nguyễn Thái Ngọc Duy wrote:\n> This is mostly for displaying the hash algorithm name when we report\n> errors. Printing \"unknown\" with '%s' is much better than '(null)' in\n> glibc printf version (and probably could crash if other implementations\n> do not check for NULL pointer)\n\nThis is a good change. Using a NULL pointer with %s on the BSDs\ndefinitely did cause a crash, last I checked.\n\nI looked at the comment I wrote in hash.h, and based on that, it is\nindeed very likely that someone would accidentally write code this way.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\nhttps://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"340188","messageId":"20180224225625.GC4620@genre.crustytoothpaste.net","threadId":"47823","inReplyTo":"20180224033429.9656-4-pclouds@gmail.com","subject":"Re: [PATCH v2 3/5] index-pack: check (and optionally set) hash algo based on input file","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2018-02-24T22:56:25Z","receivedAt":"2018-02-24T22:56:36Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Sat, Feb 24, 2018 at 10:34:27AM +0700, Nguyễn Thái Ngọc Duy wrote:\n> After 454253f059 (builtin/index-pack: improve hash function abstraction\n> - 2018-02-01), index-pack uses the_hash_algo for hashing. If \"git\n> index-pack\" is executed without a repository, we do not know what hash\n> algorithm to be used and the_hash_algo in theory could be undefined.\n> \n> Since there should be some information about the hash algorithm in the\n> input pack file, we can initialize the correct hash algorithm with that\n> if the_hash_algo is not yet initialized. This assumes that pack files\n> with new hash algorithm MUST step up pack version.\n> \n> While at there, make sure the hash algorithm requested by the pack file\n> and configured by the repository (if we're running with a repo) are\n> consistent.\n> \n> Signed-off-by: Nguyễn Thái Ngọc Duy \n> ---\n> builtin/index-pack.c | 26 +++++++++++++++++++++++++-\n> 1 file changed, 25 insertions(+), 1 deletion(-)\n> \n> diff --git a/builtin/index-pack.c b/builtin/index-pack.c\n> index 7e3e1a461c..1144458140 100644\n> --- a/builtin/index-pack.c\n> +++ b/builtin/index-pack.c\n> @@ -326,10 +326,31 @@ static const char *open_pack_file(const char *pack_name)\n> \t\toutput_fd = -1;\n> \t\tnothread_data.pack_fd = input_fd;\n> \t}\n> -\tthe_hash_algo->init_fn(&input_ctx);\n> \treturn pack_name;\n> }\n> \n> +static void prepare_hash_algo(uint32_t pack_version)\n> +{\n> +\tconst struct git_hash_algo *pack_algo;\n> +\n> +\tswitch (pack_version) {\n> +\tcase 2:\n> +\tcase 3:\n> +\t\tpack_algo = &hash_algos[GIT_HASH_SHA1];\n> +\t\tbreak;\n> +\tdefault:\n> +\t\tdie(\"BUG: how to determine hash algo for new version?\");\n> +\t}\n> +\n> +\tif (!the_hash_algo)\t/* running without repo */\n> +\t\tthe_hash_algo = pack_algo;\n> +\n> +\tif (the_hash_algo != pack_algo)\n> +\t\tdie(_(\"incompatible hash algorithm, \"\n> +\t\t \"configured for %s but the pack file needs %s\"),\n> +\t\t the_hash_algo->name, pack_algo->name);\n\nI like this. It's a nice improvement and it should be easy for us to\npass additional information into the function when our pack format\nunderstands multiple algorithms.\n\nI might have done the comparison using the format_id members instead of\nthe pointers themselves, but that's more a personal preference than\nanything.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\nhttps://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"340189","messageId":"20180224225839.GD4620@genre.crustytoothpaste.net","threadId":"47823","inReplyTo":"20180224033429.9656-6-pclouds@gmail.com","subject":"Re: [PATCH v2 5/5] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2018-02-24T22:58:39Z","receivedAt":"2018-02-24T22:58:50Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Sat, Feb 24, 2018 at 10:34:29AM +0700, Nguyễn Thái Ngọc Duy wrote:\n> This reverts commit e26f7f19b6c7485f04234946a59ab8f4fd21d6d1. The root\n> problem, git clone not setting up the_hash_algo, has been fixed in the\n> previous patch.\n> \n> Since this is a dangerous move and could potentially break stuff after\n> release (and leads to workaround like the reverted commit), the\n> workaround technically remains, but is hidden behind a new environment\n> variable GIT_HASH_FIXUP. This should let the users continue to use git\n> while we fix the problem. This variable can be deleted after one or two\n> releases.\n> \n> Signed-off-by: Nguyễn Thái Ngọc Duy \n> ---\n> common-main.c | 10 ++++++++++\n> repository.c | 2 +-\n> t/helper/test-dump-split-index.c | 2 ++\n> 3 files changed, 13 insertions(+), 1 deletion(-)\n> \n> diff --git a/common-main.c b/common-main.c\n> index 6a689007e7..fbfa98c3b8 100644\n> --- a/common-main.c\n> +++ b/common-main.c\n> @@ -1,6 +1,7 @@\n> #include \"cache.h\"\n> #include \"exec_cmd.h\"\n> #include \"attr.h\"\n> +#include \"repository.h\"\n> \n> /*\n> * Many parts of Git have subprograms communicate via pipe, expect the\n> @@ -40,5 +41,14 @@ int main(int argc, const char **argv)\n> \n> \trestore_sigpipe_to_default();\n> \n> +\t/*\n> +\t * Temporary recovery measure while hashing code is being\n> +\t * refactored. This variable should be gone after everybody\n> +\t * has used the_hash_algo in one or two releases and nobody\n> +\t * complains anything.\n> +\t */\n> +\tif (getenv(\"GIT_HASH_FIXUP\"))\n> +\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n> +\n> \treturn cmd_main(argc, argv);\n> }\n> diff --git a/repository.c b/repository.c\n> index 4ffbe9bc94..0d715f4fdb 100644\n> --- a/repository.c\n> +++ b/repository.c\n> @@ -5,7 +5,7 @@\n> \n> /* The main repository */\n> static struct repository the_repo = {\n> -\tNULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0\n> +\tNULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, NULL, 0, 0\n\nI'm wondering, now that you have the name field for the unknown value,\nif that might be a better choice here than NULL. I don't have a strong\npreference either way, so whatever you decide here is fine.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\nhttps://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"340192","messageId":"CACsJy8BP0ro6raD-s4uYOqMg1BhSEtNdn-HCpHYDz_NHXgkRbA@mail.gmail.com","threadId":"47823","inReplyTo":"20180224225839.GD4620@genre.crustytoothpaste.net","subject":"Re: [PATCH v2 5/5] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-25T03:29:29Z","receivedAt":"2018-02-25T03:33:00Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sun, Feb 25, 2018 at 5:58 AM, brian m. carlson\n wrote:\n>> diff --git a/repository.c b/repository.c\n>> index 4ffbe9bc94..0d715f4fdb 100644\n>> --- a/repository.c\n>> +++ b/repository.c\n>> @@ -5,7 +5,7 @@\n>>\n>> /* The main repository */\n>> static struct repository the_repo = {\n>> - NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0\n>> + NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, NULL, 0, 0\n>\n> I'm wondering, now that you have the name field for the unknown value,\n> if that might be a better choice here than NULL. I don't have a strong\n> preference either way, so whatever you decide here is fine.\n\nI did try that first, but for the purpose of catching uninitialized\nalgo use, NULL is better.\n\nWhen I set unknown hash algo here, I think some test failed\nmysteriously because it used rawsz field (which has value zero), it\ndidn't match some expectation, the code went to the error handling\npath, which eventually failed with some error message, but it's not\nobvious that the problem was rawsz being zero and back tracking that\ntook me some time.\n\nWith NULL hash_algo, any dereferencing fails immediately with a nice\nstack trace. Another reason to push me towards NULL hash algo is, even\nif we prefer nice messages over segmentation faults, we can't avoid it\ncompletely anyway (empty_tree and empty_blob are still NULL in unknown\nhash algo and will cause segfaults). Might as well make things\nconsistent and always segfault.\n-- \nDuy\n"},{"id":"340198","messageId":"20180225111840.16421-1-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180224033429.9656-1-pclouds@gmail.com","subject":"[PATCH v3 0/6] Fix initializing the_hash_algo","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-25T11:18:34Z","receivedAt":"2018-02-25T11:18:55Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"v3 refines v2 a bit more:\n\n- fix configure typo (and stray words in commit message)\n- use repo_set_hash_algo() instead of reassigning the_hash_algo\n- compare hash algos by format_id\n- catch NULL hash algo, report nicely and suggest GIT_HASH_FIXUP\n\nThe last point makes me much happier about keeping this workaround\naround until we are confident we can live without it. Interdiff\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 1144458140..2c75f28b41 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -331,21 +331,24 @@ static const char *open_pack_file(const char *pack_name)\n \n static void prepare_hash_algo(uint32_t pack_version)\n {\n+\tint pack_algo_id;\n \tconst struct git_hash_algo *pack_algo;\n \n \tswitch (pack_version) {\n \tcase 2:\n \tcase 3:\n-\t\tpack_algo = &hash_algos[GIT_HASH_SHA1];\n+\t\tpack_algo_id = GIT_HASH_SHA1;\n \t\tbreak;\n \tdefault:\n-\t\tdie(\"BUG: how to determine hash algo for new version?\");\n+\t\tdie(\"BUG: how to determine hash algo for version %d?\",\n+\t\t pack_version);\n \t}\n \n-\tif (!the_hash_algo)\t/* running without repo */\n-\t\tthe_hash_algo = pack_algo;\n+\tif (!repo_has_valid_hash_algo(the_repository)) /* running without repo */\n+\t\trepo_set_hash_algo(the_repository, pack_algo_id);\n \n-\tif (the_hash_algo != pack_algo)\n+\tpack_algo = &hash_algos[pack_algo_id];\n+\tif (the_hash_algo->format_id != pack_algo->format_id)\n \t\tdie(_(\"incompatible hash algorithm, \"\n \t\t \"configured for %s but the pack file needs %s\"),\n \t\t the_hash_algo->name, pack_algo->name);\ndiff --git a/cache.h b/cache.h\nindex 6b97138264..55b31e9756 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -53,7 +53,7 @@ struct object_id {\n \tunsigned char hash[GIT_MAX_RAWSZ];\n };\n \n-#define the_hash_algo the_repository->hash_algo\n+#define the_hash_algo repo_get_hash_algo(the_repository)\n \n #if defined(DT_UNKNOWN) && !defined(NO_D_TYPE_IN_DIRENT)\n #define DTYPE(de)\t((de)->d_type)\ndiff --git a/diff.c b/diff.c\nindex f5755425b6..5f28d84b87 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -3997,15 +3997,15 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)\n \n \t/*\n \t * NEEDSWORK: When running in no-index mode (and no repo is\n-\t * found, thus no hash algo conifugred), fall back to SHA-1\n+\t * found, thus no hash algo configured), fall back to SHA-1\n \t * hashing (which is used by diff_fill_oid_info below) to\n \t * avoid regression in diff output.\n \t *\n \t * In future, perhaps we can allow the user to specify their\n \t * hash algorithm from command line in this mode.\n \t */\n-\tif (o->flags.no_index && !the_hash_algo)\n-\t\tthe_hash_algo = &hash_algos[GIT_HASH_SHA1];\n+\tif (o->flags.no_index && !repo_has_valid_hash_algo(the_repository))\n+\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n \n \tdiff_fill_oid_info(one);\n \tdiff_fill_oid_info(two);\ndiff --git a/repository.h b/repository.h\nindex 0329e40c7f..40bd49611f 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -107,4 +107,20 @@ extern void repo_clear(struct repository *repo);\n */\n extern int repo_read_index(struct repository *repo);\n \n+static inline const struct git_hash_algo *repo_get_hash_algo(\n+\tconst struct repository *repo)\n+{\n+\tif (!repo->hash_algo)\n+\t\tdie(\"BUG: hash_algo is not initialized!\\n%s\",\n+\t\t _(\"You can work around this by setting environment\"\n+\t\t \" variable GIT_HASH_FIXUP=1.\\n\"\n+\t\t \"Please report this to git@vger.kernel.org\"));\n+\treturn repo->hash_algo;\n+}\n+\n+static inline int repo_has_valid_hash_algo(const struct repository *repo)\n+{\n+\treturn repo->hash_algo != NULL;\n+}\n+\n #endif /* REPOSITORY_H */\n\nNguyễn Thái Ngọc Duy (6):\n setup.c: initialize the_repository correctly in all cases\n sha1_file.c: keep a good name for \"unknown\" hash_algos[UNKNOWN]\n cache.h: make the_hash_algo read-only\n index-pack: check (and optionally set) hash algo based on input file\n diff.c: initialize hash algo when running in --no-index mode\n Revert \"repository: pre-initialize hash algo pointer\"\n\n builtin/index-pack.c | 29 ++++++++++++++++++++++++++++-\n builtin/init-db.c | 3 ++-\n cache.h | 5 +++--\n common-main.c | 10 ++++++++++\n diff.c | 12 ++++++++++++\n path.c | 2 +-\n repository.c | 2 +-\n repository.h | 16 ++++++++++++++++\n setup.c | 5 ++++-\n sha1_file.c | 2 +-\n t/helper/test-dump-split-index.c | 2 ++\n 11 files changed, 80 insertions(+), 8 deletions(-)\n\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340199","messageId":"20180225111840.16421-2-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180225111840.16421-1-pclouds@gmail.com","subject":"[PATCH v3 1/6] setup.c: initialize the_repository correctly in all cases","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-25T11:18:35Z","receivedAt":"2018-02-25T11:19:01Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"There are many ways for any command to access a git repository:\n\n- most of them will try to discover the .git dir via\n setup_git_directory() and friends\n\n- the server side programs already know where the repo is and prepare\n with enter_repo()\n\n- special commands that deal with repo creation (init/clone) use\n init_db() once the new repo is ready for access.\n\n- somebody accesses $GIT_DIR before any of above functions are called\n and accidentally sets up a git repository by set_git_dir() alone\n\n\"the_repository\" is partially set up via set_git_dir() at some point\nin all four cases. The hash algorithm though is configured later after\n.git/config is read.\n\nSo far proper repo initialization is done only for the first case [1].\nThe second case is not covered (but that's fine [3]). The third case\nwas found and worked around in [2]. The fourth case is a buggy one,\nwhich should be fixed already by jk/no-looking-at-dotgit-outside-repo\nand never happens again.\n\nThis patch makes sure all cases initialize the hash algorithm in\nthe_repository correctly. Both second and third cases must run\ncheck_repo_format() before \"entering\" it. Eventually we probably just\nrename this function to init_repo() or something.\n\n[1] 78a6766802 (Integrate hash algorithm support with repo setup -\n 2017-11-12)\n\n[2] e26f7f19b6 (repository: pre-initialize hash algo pointer -\n 2018-01-19)\n\n[3] the reason server side is still running ok with no hash algo before\n [2] is because the programs that use enter_repo() do very\n little (and unlikely to hash anything) then spawn a new\n program (like pack-objects or upload-archive) to do the heavy\n lifting. These programs already use setup_git_directory() or the\n gently version\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/init-db.c | 3 ++-\n cache.h | 3 ++-\n path.c | 2 +-\n setup.c | 5 ++++-\n 4 files changed, 9 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex c9b7946bad..d119d9906b 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -9,6 +9,7 @@\n #include \"builtin.h\"\n #include \"exec_cmd.h\"\n #include \"parse-options.h\"\n+#include \"repository.h\"\n \n #ifndef DEFAULT_GIT_TEMPLATE_DIR\n #define DEFAULT_GIT_TEMPLATE_DIR \"/usr/share/git-core/templates\"\n@@ -369,7 +370,7 @@ int init_db(const char *git_dir, const char *real_git_dir,\n \t * config file, so this will not fail. What we are catching\n \t * is an attempt to reinitialize new repository with an old tool.\n \t */\n-\tcheck_repository_format();\n+\tcheck_repository_format(the_repository);\n \n \treinit = create_default_files(template_dir, original_git_dir);\n \ndiff --git a/cache.h b/cache.h\nindex 21fbcc2414..6b97138264 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -894,6 +894,7 @@ extern int repository_format_precious_objects;\n extern char *repository_format_partial_clone;\n extern const char *core_partial_clone_filter_default;\n \n+struct repository;\n struct repository_format {\n \tint version;\n \tint precious_objects;\n@@ -926,7 +927,7 @@ int verify_repository_format(const struct repository_format *format,\n * set_git_dir() before calling this, and use it only for \"are we in a valid\n * repo?\".\n */\n-extern void check_repository_format(void);\n+extern void check_repository_format(struct repository *);\n \n #define MTIME_CHANGED\t0x0001\n #define CTIME_CHANGED\t0x0002\ndiff --git a/path.c b/path.c\nindex da8b655730..a544252198 100644\n--- a/path.c\n+++ b/path.c\n@@ -827,7 +827,7 @@ const char *enter_repo(const char *path, int strict)\n \n \tif (is_git_directory(\".\")) {\n \t\tset_git_dir(\".\");\n-\t\tcheck_repository_format();\n+\t\tcheck_repository_format(the_repository);\n \t\treturn path;\n \t}\n \ndiff --git a/setup.c b/setup.c\nindex c5d55dcee4..a82103832e 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -1180,11 +1180,14 @@ int git_config_perm(const char *var, const char *value)\n \treturn -(i & 0666);\n }\n \n-void check_repository_format(void)\n+/* optionally configure \"repo\" to the correct format */\n+void check_repository_format(struct repository *repo)\n {\n \tstruct repository_format repo_fmt;\n \tcheck_repository_format_gently(get_git_dir(), &repo_fmt, NULL);\n \tstartup_info->have_repository = 1;\n+\tif (repo)\n+\t\trepo_set_hash_algo(repo, repo_fmt.hash_algo);\n }\n \n /*\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340200","messageId":"20180225111840.16421-3-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180225111840.16421-1-pclouds@gmail.com","subject":"[PATCH v3 2/6] sha1_file.c: keep a good name for \"unknown\" hash_algos[UNKNOWN]","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-25T11:18:36Z","receivedAt":"2018-02-25T11:19:06Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This is mostly for displaying the hash algorithm name when we report\nerrors. Printing \"unknown\" with '%s' is much better than '(null)' in\nglibc printf version (and probably could crash if other implementations\ndo not check for NULL pointer)\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n sha1_file.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 826d7a0ae3..4b266b1c68 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -71,7 +71,7 @@ static void git_hash_unknown_final(unsigned char *hash, git_hash_ctx *ctx)\n \n const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = {\n \t{\n-\t\tNULL,\n+\t\t\"unknown\",\n \t\t0x00000000,\n \t\t0,\n \t\t0,\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340201","messageId":"20180225111840.16421-4-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180225111840.16421-1-pclouds@gmail.com","subject":"[PATCH v3 3/6] cache.h: make the_hash_algo read-only","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-25T11:18:37Z","receivedAt":"2018-02-25T11:19:12Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"By returning an R-value in the_hash_algo we make sure people can't\naccidentally change hash algorithm with\n\n the_hash_algo = &hash_algos[something];\n\nand go with repo_set_hash_algo() instead. Of course they can still do\n\n the_repository->hash_algo = ...\n\nbut that is more obvious and easily caught in review.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n cache.h | 2 +-\n repository.h | 6 ++++++\n 2 files changed, 7 insertions(+), 1 deletion(-)\n\ndiff --git a/cache.h b/cache.h\nindex 6b97138264..55b31e9756 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -53,7 +53,7 @@ struct object_id {\n \tunsigned char hash[GIT_MAX_RAWSZ];\n };\n \n-#define the_hash_algo the_repository->hash_algo\n+#define the_hash_algo repo_get_hash_algo(the_repository)\n \n #if defined(DT_UNKNOWN) && !defined(NO_D_TYPE_IN_DIRENT)\n #define DTYPE(de)\t((de)->d_type)\ndiff --git a/repository.h b/repository.h\nindex 0329e40c7f..5092df3700 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -107,4 +107,10 @@ extern void repo_clear(struct repository *repo);\n */\n extern int repo_read_index(struct repository *repo);\n \n+static inline const struct git_hash_algo *repo_get_hash_algo(\n+\tconst struct repository *repo)\n+{\n+\treturn repo->hash_algo;\n+}\n+\n #endif /* REPOSITORY_H */\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340202","messageId":"20180225111840.16421-5-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180225111840.16421-1-pclouds@gmail.com","subject":"[PATCH v3 4/6] index-pack: check (and optionally set) hash algo based on input file","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-25T11:18:38Z","receivedAt":"2018-02-25T11:19:16Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"After 454253f059 (builtin/index-pack: improve hash function abstraction\n- 2018-02-01), index-pack uses the_hash_algo for hashing. If \"git\nindex-pack\" is executed without a repository, we do not know what hash\nalgorithm to be used and the_hash_algo in theory could be undefined.\n\nSince there should be some information about the hash algorithm in the\ninput pack file, we can initialize the correct hash algorithm with that\nif the_hash_algo is not yet initialized. This assumes that pack files\nwith new hash algorithm MUST step up pack version.\n\nWhile at there, make sure the hash algorithm requested by the pack file\nand configured by the repository (if we're running with a repo) are\nconsistent.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n builtin/index-pack.c | 29 ++++++++++++++++++++++++++++-\n repository.h | 5 +++++\n 2 files changed, 33 insertions(+), 1 deletion(-)\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 7e3e1a461c..2c75f28b41 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -326,10 +326,34 @@ static const char *open_pack_file(const char *pack_name)\n \t\toutput_fd = -1;\n \t\tnothread_data.pack_fd = input_fd;\n \t}\n-\tthe_hash_algo->init_fn(&input_ctx);\n \treturn pack_name;\n }\n \n+static void prepare_hash_algo(uint32_t pack_version)\n+{\n+\tint pack_algo_id;\n+\tconst struct git_hash_algo *pack_algo;\n+\n+\tswitch (pack_version) {\n+\tcase 2:\n+\tcase 3:\n+\t\tpack_algo_id = GIT_HASH_SHA1;\n+\t\tbreak;\n+\tdefault:\n+\t\tdie(\"BUG: how to determine hash algo for version %d?\",\n+\t\t pack_version);\n+\t}\n+\n+\tif (!repo_has_valid_hash_algo(the_repository)) /* running without repo */\n+\t\trepo_set_hash_algo(the_repository, pack_algo_id);\n+\n+\tpack_algo = &hash_algos[pack_algo_id];\n+\tif (the_hash_algo->format_id != pack_algo->format_id)\n+\t\tdie(_(\"incompatible hash algorithm, \"\n+\t\t \"configured for %s but the pack file needs %s\"),\n+\t\t the_hash_algo->name, pack_algo->name);\n+}\n+\n static void parse_pack_header(void)\n {\n \tstruct pack_header *hdr = fill(sizeof(struct pack_header));\n@@ -341,6 +365,9 @@ static void parse_pack_header(void)\n \t\tdie(_(\"pack version %\"PRIu32\" unsupported\"),\n \t\t\tntohl(hdr->hdr_version));\n \n+\tprepare_hash_algo(ntohl(hdr->hdr_version));\n+\tthe_hash_algo->init_fn(&input_ctx);\n+\n \tnr_objects = ntohl(hdr->hdr_entries);\n \tuse(sizeof(struct pack_header));\n }\ndiff --git a/repository.h b/repository.h\nindex 5092df3700..02c695bc74 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -113,4 +113,9 @@ static inline const struct git_hash_algo *repo_get_hash_algo(\n \treturn repo->hash_algo;\n }\n \n+static inline int repo_has_valid_hash_algo(const struct repository *repo)\n+{\n+\treturn repo->hash_algo != NULL;\n+}\n+\n #endif /* REPOSITORY_H */\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340203","messageId":"20180225111840.16421-6-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180225111840.16421-1-pclouds@gmail.com","subject":"[PATCH v3 5/6] diff.c: initialize hash algo when running in --no-index mode","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-25T11:18:39Z","receivedAt":"2018-02-25T11:19:22Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Our \"git diff\" command supports running as a standalone tool. In this\ncode path, we try to hash the file content but after\n18e2588e11 (sha1_file: switch uses of SHA-1 to the_hash_algo -\n2018-02-01), there is a chance that the_hash_algo (required by\nindex_path) may still be uninitialized if no repository is found.\n\nExecuting index_path() when the_hash_algo is NULL (or points to unknown\nalgo) either crashes or dies. Let's make it a bit safer by explicitly\nfalling back to SHA-1 (so that the diff output remains the same as\nbefore, compared to the alternative that we simply do not hash).\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n diff.c | 12 ++++++++++++\n 1 file changed, 12 insertions(+)\n\ndiff --git a/diff.c b/diff.c\nindex 21c3838b25..5f28d84b87 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -3995,6 +3995,18 @@ static void run_diff(struct diff_filepair *p, struct diff_options *o)\n \t\treturn;\n \t}\n \n+\t/*\n+\t * NEEDSWORK: When running in no-index mode (and no repo is\n+\t * found, thus no hash algo configured), fall back to SHA-1\n+\t * hashing (which is used by diff_fill_oid_info below) to\n+\t * avoid regression in diff output.\n+\t *\n+\t * In future, perhaps we can allow the user to specify their\n+\t * hash algorithm from command line in this mode.\n+\t */\n+\tif (o->flags.no_index && !repo_has_valid_hash_algo(the_repository))\n+\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n+\n \tdiff_fill_oid_info(one);\n \tdiff_fill_oid_info(two);\n \n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340204","messageId":"20180225111840.16421-7-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180225111840.16421-1-pclouds@gmail.com","subject":"[PATCH v3 6/6] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-25T11:18:40Z","receivedAt":"2018-02-25T11:19:29Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This reverts commit e26f7f19b6c7485f04234946a59ab8f4fd21d6d1. The root\nproblem, git clone not setting up the_hash_algo, has been fixed in the\nprevious patch.\n\nSince this is a dangerous move and could potentially break stuff after\nrelease (and leads to workaround like the reverted commit), the\nworkaround technically remains, but is hidden behind a new environment\nvariable GIT_HASH_FIXUP. This should let the users continue to use git\nwhile we fix the problem. This variable can be deleted after one or two\nreleases.\n\ntest-dump-split-index.c needs to call setup_git_directory() after this\nto configure hash algorithm before parsing the index file. It's not the\nbest way (i.e. check index file version) but it's a test tool, this is\ngood enough.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n common-main.c | 10 ++++++++++\n repository.c | 2 +-\n repository.h | 5 +++++\n t/helper/test-dump-split-index.c | 2 ++\n 4 files changed, 18 insertions(+), 1 deletion(-)\n\ndiff --git a/common-main.c b/common-main.c\nindex 6a689007e7..fbfa98c3b8 100644\n--- a/common-main.c\n+++ b/common-main.c\n@@ -1,6 +1,7 @@\n #include \"cache.h\"\n #include \"exec_cmd.h\"\n #include \"attr.h\"\n+#include \"repository.h\"\n \n /*\n * Many parts of Git have subprograms communicate via pipe, expect the\n@@ -40,5 +41,14 @@ int main(int argc, const char **argv)\n \n \trestore_sigpipe_to_default();\n \n+\t/*\n+\t * Temporary recovery measure while hashing code is being\n+\t * refactored. This variable should be gone after everybody\n+\t * has used the_hash_algo in one or two releases and nobody\n+\t * complains anything.\n+\t */\n+\tif (getenv(\"GIT_HASH_FIXUP\"))\n+\t\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n+\n \treturn cmd_main(argc, argv);\n }\ndiff --git a/repository.c b/repository.c\nindex 4ffbe9bc94..0d715f4fdb 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -5,7 +5,7 @@\n \n /* The main repository */\n static struct repository the_repo = {\n-\tNULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0\n+\tNULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, NULL, 0, 0\n };\n struct repository *the_repository = &the_repo;\n \ndiff --git a/repository.h b/repository.h\nindex 02c695bc74..40bd49611f 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -110,6 +110,11 @@ extern int repo_read_index(struct repository *repo);\n static inline const struct git_hash_algo *repo_get_hash_algo(\n \tconst struct repository *repo)\n {\n+\tif (!repo->hash_algo)\n+\t\tdie(\"BUG: hash_algo is not initialized!\\n%s\",\n+\t\t _(\"You can work around this by setting environment\"\n+\t\t \" variable GIT_HASH_FIXUP=1.\\n\"\n+\t\t \"Please report this to git@vger.kernel.org\"));\n \treturn repo->hash_algo;\n }\n \ndiff --git a/t/helper/test-dump-split-index.c b/t/helper/test-dump-split-index.c\nindex e44430b699..b759fe3fc1 100644\n--- a/t/helper/test-dump-split-index.c\n+++ b/t/helper/test-dump-split-index.c\n@@ -12,6 +12,8 @@ int cmd_main(int ac, const char **av)\n \tstruct split_index *si;\n \tint i;\n \n+\tsetup_git_directory();\n+\n \tdo_read_index(&the_index, av[1], 1);\n \tprintf(\"own %s\\n\", sha1_to_hex(the_index.sha1));\n \tsi = the_index.split_index;\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340230","messageId":"20180225202816.GE4620@genre.crustytoothpaste.net","threadId":"47823","inReplyTo":"CACsJy8BP0ro6raD-s4uYOqMg1BhSEtNdn-HCpHYDz_NHXgkRbA@mail.gmail.com","subject":"Re: [PATCH v2 5/5] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2018-02-25T20:28:16Z","receivedAt":"2018-02-25T20:28:32Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Sun, Feb 25, 2018 at 10:29:29AM +0700, Duy Nguyen wrote:\n> When I set unknown hash algo here, I think some test failed\n> mysteriously because it used rawsz field (which has value zero), it\n> didn't match some expectation, the code went to the error handling\n> path, which eventually failed with some error message, but it's not\n> obvious that the problem was rawsz being zero and back tracking that\n> took me some time.\n> \n> With NULL hash_algo, any dereferencing fails immediately with a nice\n> stack trace. Another reason to push me towards NULL hash algo is, even\n> if we prefer nice messages over segmentation faults, we can't avoid it\n> completely anyway (empty_tree and empty_blob are still NULL in unknown\n> hash algo and will cause segfaults). Might as well make things\n> consistent and always segfault.\n\nThat makes sense. Thanks for the explanation.\n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\nhttps://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"340231","messageId":"20180225203401.GF4620@genre.crustytoothpaste.net","threadId":"47823","inReplyTo":"20180225111840.16421-1-pclouds@gmail.com","subject":"Re: [PATCH v3 0/6] Fix initializing the_hash_algo","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2018-02-25T20:34:01Z","receivedAt":"2018-02-25T20:34:12Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Sun, Feb 25, 2018 at 06:18:34PM +0700, Nguyễn Thái Ngọc Duy wrote:\n> v3 refines v2 a bit more:\n> \n> - fix configure typo (and stray words in commit message)\n> - use repo_set_hash_algo() instead of reassigning the_hash_algo\n> - compare hash algos by format_id\n> - catch NULL hash algo, report nicely and suggest GIT_HASH_FIXUP\n> \n> The last point makes me much happier about keeping this workaround\n> around until we are confident we can live without it. Interdiff\n\nThis looks sane to me.\n\nReviewed-by: brian m. carlson \n-- \nbrian m. carlson / brian with sandals: Houston, Texas, US\nhttps://www.crustytoothpaste.net/~bmc | My opinion only\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"340275","messageId":"20180226085508.GA30343@sigill.intra.peff.net","threadId":"47823","inReplyTo":"CAGZ79kahS_mk8gFONUAxBjw7iV=5q1ff-ZgjLnQ7rMw=kBKThw@mail.gmail.com","subject":"Re: [PATCH 08/27] pack: move approximate object count to object store","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-02-26T08:55:08Z","receivedAt":"2018-02-26T08:55:15Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Feb 23, 2018 at 02:22:14PM -0800, Stefan Beller wrote:\n\n> >> + /*\n> >> + * A fast, rough count of the number of objects in the repository.\n> >> + * These two fields are not meant for direct access. Use\n> >> + * approximate_object_count() instead.\n> >> + */\n> >> + unsigned long approximate_object_count;\n> >> + unsigned approximate_object_count_valid : 1;\n> >\n> > Patch looks fine and is effectively a no-op, though what is the need for\n> > both of these variables? Maybe it can be simplified down to just use\n> > one? Just musing as its out of the scope of this patch and we probably\n> > shouldn't try to change that in this series.\n> \n> I agree we should. It was introduced in e323de4ad7f (sha1_file:\n> allow sha1_loose_object_info to handle arbitrary repositories, 2017-08-30)\n> and I think it was seen as a clever optimization trick back then?\n\nI think you meant 8e3f52d778 (find_unique_abbrev: move logic out of\nget_short_sha1(), 2016-10-03)?\n\nYes, it was just to avoid the dual-meaning of \"0\" for \"not set\" and a\nrepository with no packfiles. It would probably be fine to get rid of\nit. If you have no packfiles then you probably don't have enough objects\nto worry about micro-optimizing. And anyway, the \"wasted\" case wouldn't\neven make any syscalls (it would do a noop prepare_packed_git() and then\nrealize the packed_git list is empty).\n\n-Peff\n"},{"id":"340278","messageId":"20180226093040.GA10479@ash","threadId":"47823","inReplyTo":"20180224004754.129721-2-sbeller@google.com","subject":"Re: [PATCHv4 01/27] repository: introduce raw object store field","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-26T09:30:41Z","receivedAt":"2018-02-26T09:30:55Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Fri, Feb 23, 2018 at 04:47:28PM -0800, Stefan Beller wrote:\n> /* The main repository */\n> static struct repository the_repo = {\n> -\tNULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0\n> +\tNULL, NULL,\n> +\tRAW_OBJECT_STORE_INIT,\n> +\tNULL, NULL, NULL,\n> +\tNULL, NULL, NULL,\n> +\t&the_index,\n> +\t&hash_algos[GIT_HASH_SHA1],\n> +\t0, 0\n> };\n> struct repository *the_repository = &the_repo;\n\nI wonder if we should do something like this. It makes the_repo\ninitialization easier to read and it helps unify the init code with\nsubmodule.\n\nNo don't reroll. If you think it's a good idea, you can do something\nlike this in the next series instead.\n\n-- 8< --\ndiff --git a/check-racy.c b/check-racy.c\nindex 24b6542352..47cbb4eb6d 100644\n--- a/check-racy.c\n+++ b/check-racy.c\n@@ -1,10 +1,12 @@\n #include \"cache.h\"\n+#include \"repository.h\"\n \n int main(int ac, char **av)\n {\n \tint i;\n \tint dirty, clean, racy;\n \n+\tinit_the_repository();\n \tdirty = clean = racy = 0;\n \tread_cache();\n \tfor (i = 0; i < active_nr; i++) {\ndiff --git a/common-main.c b/common-main.c\nindex 6a689007e7..a13ab981aa 100644\n--- a/common-main.c\n+++ b/common-main.c\n@@ -1,6 +1,7 @@\n #include \"cache.h\"\n #include \"exec_cmd.h\"\n #include \"attr.h\"\n+#include \"repository.h\"\n \n /*\n * Many parts of Git have subprograms communicate via pipe, expect the\n@@ -32,6 +33,8 @@ int main(int argc, const char **argv)\n \t */\n \tsanitize_stdfds();\n \n+\tinit_the_repository();\n+\n \tgit_setup_gettext();\n \n \tattr_start();\ndiff --git a/object-store.h b/object-store.h\nindex cf35760ceb..c3253ebc59 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -8,8 +8,8 @@ struct raw_object_store {\n \t */\n \tchar *objectdir;\n };\n-#define RAW_OBJECT_STORE_INIT { NULL }\n \n+void raw_object_store_init(struct raw_object_store *o);\n void raw_object_store_clear(struct raw_object_store *o);\n \n #endif /* OBJECT_STORE_H */\ndiff --git a/object.c b/object.c\nindex 11d904c033..8a4d01dd5f 100644\n--- a/object.c\n+++ b/object.c\n@@ -446,6 +446,11 @@ void clear_commit_marks_all(unsigned int flags)\n \t}\n }\n \n+void raw_object_store_init(struct raw_object_store *o)\n+{\n+\tmemset(o, 0, sizeof(*o));\n+}\n+\n void raw_object_store_clear(struct raw_object_store *o)\n {\n \tfree(o->objectdir);\ndiff --git a/repository.c b/repository.c\nindex 2255ff657e..0ebcca8539 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -5,16 +5,22 @@\n #include \"submodule-config.h\"\n \n /* The main repository */\n-static struct repository the_repo = {\n-\tNULL, NULL,\n-\tRAW_OBJECT_STORE_INIT,\n-\tNULL, NULL, NULL,\n-\tNULL, NULL, NULL,\n-\t&the_index,\n-\t&hash_algos[GIT_HASH_SHA1],\n-\t0, 0\n-};\n-struct repository *the_repository = &the_repo;\n+static struct repository the_repo;\n+struct repository *the_repository;\n+\n+static void repo_pre_init(struct repository *repo)\n+{\n+\tmemset(repo, 0, sizeof(*repo));\n+\traw_object_store_init(&repo->objects);\n+}\n+\n+void init_the_repository(void)\n+{\n+\tthe_repository = &the_repo;\n+\trepo_pre_init(the_repository);\n+\tthe_repository->index = &the_index;\n+\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n+}\n \n static char *git_path_from_env(const char *envvar, const char *git_dir,\n \t\t\t const char *path, int fromenv)\n@@ -138,7 +144,8 @@ static int read_and_verify_repository_format(struct repository_format *format,\n int repo_init(struct repository *repo, const char *gitdir, const char *worktree)\n {\n \tstruct repository_format format;\n-\tmemset(repo, 0, sizeof(*repo));\n+\n+\trepo_pre_init(repo);\n \n \trepo->ignore_env = 1;\n \ndiff --git a/repository.h b/repository.h\nindex 1f8bc7a7cf..da6a8f9af9 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -93,6 +93,7 @@ extern void repo_set_gitdir(struct repository *repo, const char *path);\n extern void repo_set_worktree(struct repository *repo, const char *path);\n extern void repo_set_hash_algo(struct repository *repo, int algo);\n extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);\n+extern void init_the_repository(void);\n extern int repo_submodule_init(struct repository *submodule,\n \t\t\t struct repository *superproject,\n \t\t\t const char *path);\n-- 8< --\n"},{"id":"340282","messageId":"20180226103030.26900-1-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180214180814.GA139458@google.com","subject":"[PATCH 0/4] Delete ignore_env member in struct repository","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-26T10:30:26Z","receivedAt":"2018-02-26T10:30:51Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"It turns out I don't need my other series [1] in order to delete this\nfield. This series moves getenv() calls from\nrepo_set_gitdir()/repo_setup_env() and prepare_alt_odb() back in\nenvironment.c where they belong in my opinion.\n\nThe repo_set_gitdir() now takes $GIT_DIR and optionally all other\nconfigurable paths. If those paths are NULL, default repo layout will\nbe used. With getenv() no longer called inside repo_set_gitdir(),\nignore_env has no reason to stay. This is in 1/4.\n\nThe getenv() in prepare_alt_odb() is also moved back to\nsetup_git_env() in 3/4. It demonstrates how we could move other\ngetenv() back to if we want.\n\nThis series is built on top of Stefan's object-store-part1, v4. I\ncould rebase it on 'master' too, but then Junio may need to resolve\nsome conflicts.\n\n[1] https://public-inbox.org/git/20180225111840.16421-1-pclouds@gmail.com/\n\nNguyễn Thái Ngọc Duy (4):\n repository.c: move env-related setup code back to environment.c\n repository.c: delete dead functions\n sha1_file.c: move delayed getenv(altdb) back to setup_git_env()\n repository: delete ignore_env member\n\n cache.h | 2 +-\n environment.c | 13 +++++++--\n object-store.h | 5 +++-\n object.c | 1 +\n repository.c | 79 ++++++++++++++++++++++----------------------------\n repository.h | 21 +++++++-------\n setup.c | 3 +-\n sha1_file.c | 6 +---\n 8 files changed, 64 insertions(+), 66 deletions(-)\n\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340283","messageId":"20180226103030.26900-2-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180226103030.26900-1-pclouds@gmail.com","subject":"[PATCH 1/4] repository.c: move env-related setup code back to environment.c","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-26T10:30:27Z","receivedAt":"2018-02-26T10:30:55Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"It does not make sense that generic repository code contains handling\nof environment variables, which are specific for the main repository\nonly. Refactor repo_set_gitdir() function to take $GIT_DIR and\noptionally _all_ other customizable paths. These optional paths can be\nNULL and will be calculated according to the default directory layout.\n\nNote that some dead functions are left behind to reduce diff\nnoise. They will be deleted in the next patch.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n cache.h | 2 +-\n environment.c | 12 +++++++++---\n repository.c | 48 ++++++++++++++++++++++++++++++++++++++++++------\n repository.h | 11 ++++++++++-\n setup.c | 3 +--\n 5 files changed, 63 insertions(+), 13 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 5717399183..b164a407eb 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -459,7 +459,7 @@ static inline enum object_type object_type(unsigned int mode)\n */\n extern const char * const local_repo_env[];\n \n-extern void setup_git_env(void);\n+extern void setup_git_env(const char *git_dir);\n \n /*\n * Returns true iff we have a configured git repository (either via\ndiff --git a/environment.c b/environment.c\nindex ec10b062e6..2905042b16 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -148,10 +148,17 @@ static char *expand_namespace(const char *raw_namespace)\n \treturn strbuf_detach(&buf, NULL);\n }\n \n-void setup_git_env(void)\n+void setup_git_env(const char *git_dir)\n {\n \tconst char *shallow_file;\n \tconst char *replace_ref_base;\n+\tstruct set_gitdir_args args = { NULL };\n+\n+\targs.shared_root = getenv(GIT_COMMON_DIR_ENVIRONMENT);\n+\targs.object_dir = getenv(DB_ENVIRONMENT);\n+\targs.graft_file = getenv(GRAFT_ENVIRONMENT);\n+\targs.index_file = getenv(INDEX_ENVIRONMENT);\n+\trepo_set_gitdir(the_repository, git_dir, &args);\n \n \tif (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))\n \t\tcheck_replace_refs = 0;\n@@ -301,8 +308,7 @@ int set_git_dir(const char *path)\n {\n \tif (setenv(GIT_DIR_ENVIRONMENT, path, 1))\n \t\treturn error(\"Could not set GIT_DIR to '%s'\", path);\n-\trepo_set_gitdir(the_repository, path);\n-\tsetup_git_env();\n+\tsetup_git_env(path);\n \treturn 0;\n }\n \ndiff --git a/repository.c b/repository.c\nindex a069b1b640..70dc8dc661 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -61,15 +61,50 @@ static void repo_setup_env(struct repository *repo)\n \t\t\t\t\t \"index\", !repo->ignore_env);\n }\n \n-void repo_set_gitdir(struct repository *repo, const char *path)\n+static void expand_base_dir(char **out, const char *in,\n+\t\t\t const char *base_dir, const char *def_in)\n {\n-\tconst char *gitfile = read_gitfile(path);\n-\tchar *old_gitdir = repo->gitdir;\n+\tfree(*out);\n+\tif (in)\n+\t\t*out = xstrdup(in);\n+\telse\n+\t\t*out = xstrfmt(\"%s/%s\", base_dir, def_in);\n+}\n+\n+static void repo_set_commondir(struct repository *repo,\n+\t\t\t const char *shared_root)\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\n+\tfree(repo->commondir);\n \n-\trepo->gitdir = xstrdup(gitfile ? gitfile : path);\n-\trepo_setup_env(repo);\n+\tif (shared_root) {\n+\t\trepo->different_commondir = 1;\n+\t\trepo->commondir = xstrdup(shared_root);\n+\t\treturn;\n+\t}\n \n+\trepo->different_commondir = get_common_dir_noenv(&sb, repo->gitdir);\n+\trepo->commondir = strbuf_detach(&sb, NULL);\n+}\n+\n+void repo_set_gitdir(struct repository *repo,\n+\t\t const char *root,\n+\t\t const struct set_gitdir_args *o)\n+{\n+\tconst char *gitfile = read_gitfile(root);\n+\tchar *old_gitdir = repo->gitdir;\n+\n+\trepo->gitdir = xstrdup(gitfile ? gitfile : root);\n \tfree(old_gitdir);\n+\n+\trepo_set_commondir(repo, o->shared_root);\n+\texpand_base_dir(&repo->objects.objectdir, o->object_dir,\n+\t\t\trepo->commondir, \"objects\");\n+\texpand_base_dir(&repo->graft_file, o->graft_file,\n+\t\t\trepo->commondir, \"info/grafts\");\n+\texpand_base_dir(&repo->index_file, o->index_file,\n+\t\t\trepo->gitdir, \"index\");\n }\n \n void repo_set_hash_algo(struct repository *repo, int hash_algo)\n@@ -87,6 +122,7 @@ static int repo_init_gitdir(struct repository *repo, const char *gitdir)\n \tint error = 0;\n \tchar *abspath = NULL;\n \tconst char *resolved_gitdir;\n+\tstruct set_gitdir_args args = { NULL };\n \n \tabspath = real_pathdup(gitdir, 0);\n \tif (!abspath) {\n@@ -101,7 +137,7 @@ static int repo_init_gitdir(struct repository *repo, const char *gitdir)\n \t\tgoto out;\n \t}\n \n-\trepo_set_gitdir(repo, resolved_gitdir);\n+\trepo_set_gitdir(repo, resolved_gitdir, &args);\n \n out:\n \tfree(abspath);\ndiff --git a/repository.h b/repository.h\nindex fa73ab8e93..b5b5d138aa 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -89,7 +89,16 @@ struct repository {\n \n extern struct repository *the_repository;\n \n-extern void repo_set_gitdir(struct repository *repo, const char *path);\n+struct set_gitdir_args {\n+\tconst char *shared_root;\n+\tconst char *object_dir;\n+\tconst char *graft_file;\n+\tconst char *index_file;\n+};\n+\n+extern void repo_set_gitdir(struct repository *repo,\n+\t\t\t const char *root,\n+\t\t\t const struct set_gitdir_args *optional);\n extern void repo_set_worktree(struct repository *repo, const char *path);\n extern void repo_set_hash_algo(struct repository *repo, int algo);\n extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);\ndiff --git a/setup.c b/setup.c\nindex c5d55dcee4..6fac1bb58a 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -1116,8 +1116,7 @@ const char *setup_git_directory_gently(int *nongit_ok)\n \t\t\tconst char *gitdir = getenv(GIT_DIR_ENVIRONMENT);\n \t\t\tif (!gitdir)\n \t\t\t\tgitdir = DEFAULT_GIT_DIR_ENVIRONMENT;\n-\t\t\trepo_set_gitdir(the_repository, gitdir);\n-\t\t\tsetup_git_env();\n+\t\t\tsetup_git_env(gitdir);\n \t\t}\n \t\tif (startup_info->have_repository)\n \t\t\trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340284","messageId":"20180226103030.26900-3-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180226103030.26900-1-pclouds@gmail.com","subject":"[PATCH 2/4] repository.c: delete dead functions","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-26T10:30:28Z","receivedAt":"2018-02-26T10:31:01Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy \n---\n repository.c | 45 ---------------------------------------------\n 1 file changed, 45 deletions(-)\n\ndiff --git a/repository.c b/repository.c\nindex 70dc8dc661..be86da82cc 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -16,51 +16,6 @@ static struct repository the_repo = {\n };\n struct repository *the_repository = &the_repo;\n \n-static char *git_path_from_env(const char *envvar, const char *git_dir,\n-\t\t\t const char *path, int fromenv)\n-{\n-\tif (fromenv) {\n-\t\tconst char *value = getenv(envvar);\n-\t\tif (value)\n-\t\t\treturn xstrdup(value);\n-\t}\n-\n-\treturn xstrfmt(\"%s/%s\", git_dir, path);\n-}\n-\n-static int find_common_dir(struct strbuf *sb, const char *gitdir, int fromenv)\n-{\n-\tif (fromenv) {\n-\t\tconst char *value = getenv(GIT_COMMON_DIR_ENVIRONMENT);\n-\t\tif (value) {\n-\t\t\tstrbuf_addstr(sb, value);\n-\t\t\treturn 1;\n-\t\t}\n-\t}\n-\n-\treturn get_common_dir_noenv(sb, gitdir);\n-}\n-\n-static void repo_setup_env(struct repository *repo)\n-{\n-\tstruct strbuf sb = STRBUF_INIT;\n-\n-\trepo->different_commondir = find_common_dir(&sb, repo->gitdir,\n-\t\t\t\t\t\t !repo->ignore_env);\n-\tfree(repo->commondir);\n-\trepo->commondir = strbuf_detach(&sb, NULL);\n-\traw_object_store_clear(&repo->objects);\n-\trepo->objects.objectdir =\n-\t\tgit_path_from_env(DB_ENVIRONMENT, repo->commondir,\n-\t\t\t\t \"objects\", !repo->ignore_env);\n-\tfree(repo->graft_file);\n-\trepo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,\n-\t\t\t\t\t \"info/grafts\", !repo->ignore_env);\n-\tfree(repo->index_file);\n-\trepo->index_file = git_path_from_env(INDEX_ENVIRONMENT, repo->gitdir,\n-\t\t\t\t\t \"index\", !repo->ignore_env);\n-}\n-\n static void expand_base_dir(char **out, const char *in,\n \t\t\t const char *base_dir, const char *def_in)\n {\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340285","messageId":"20180226103030.26900-4-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180226103030.26900-1-pclouds@gmail.com","subject":"[PATCH 3/4] sha1_file.c: move delayed getenv(altdb) back to setup_git_env()","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-26T10:30:29Z","receivedAt":"2018-02-26T10:31:06Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"getenv() is supposed to work on the main repository only. This delayed\ngetenv() code in sha1_file.c makes it more difficult to convert\nsha1_file.c to a generic object store that could be used by both\nsubmodule and main repositories.\n\nMove the getenv() back in setup_git_env() where other env vars are\nalso fetched.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n environment.c | 1 +\n object-store.h | 5 ++++-\n object.c | 1 +\n repository.c | 2 ++\n repository.h | 1 +\n sha1_file.c | 6 +-----\n 6 files changed, 10 insertions(+), 6 deletions(-)\n\ndiff --git a/environment.c b/environment.c\nindex 2905042b16..95de419de8 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -158,6 +158,7 @@ void setup_git_env(const char *git_dir)\n \targs.object_dir = getenv(DB_ENVIRONMENT);\n \targs.graft_file = getenv(GRAFT_ENVIRONMENT);\n \targs.index_file = getenv(INDEX_ENVIRONMENT);\n+\targs.alternate_db = getenv(ALTERNATE_DB_ENVIRONMENT);\n \trepo_set_gitdir(the_repository, git_dir, &args);\n \n \tif (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))\ndiff --git a/object-store.h b/object-store.h\nindex afe2f93459..9b1303549b 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -87,6 +87,9 @@ struct raw_object_store {\n \t */\n \tchar *objectdir;\n \n+\t/* Path to extra alternate object database if not NULL */\n+\tchar *alternate_db;\n+\n \tstruct packed_git *packed_git;\n \t/* A most-recently-used ordered version of the packed_git list. */\n \tstruct list_head packed_git_mru;\n@@ -109,7 +112,7 @@ struct raw_object_store {\n \tunsigned packed_git_initialized : 1;\n };\n \n-#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0, 0, 0 }\n+#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0, 0, 0 }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/object.c b/object.c\nindex a7c238339b..5317cfc390 100644\n--- a/object.c\n+++ b/object.c\n@@ -464,6 +464,7 @@ static void free_alt_odbs(struct raw_object_store *o)\n void raw_object_store_clear(struct raw_object_store *o)\n {\n \tFREE_AND_NULL(o->objectdir);\n+\tFREE_AND_NULL(o->alternate_db);\n \n \tfree_alt_odbs(o);\n \to->alt_odb_tail = NULL;\ndiff --git a/repository.c b/repository.c\nindex be86da82cc..19532ec5ff 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -56,6 +56,8 @@ void repo_set_gitdir(struct repository *repo,\n \trepo_set_commondir(repo, o->shared_root);\n \texpand_base_dir(&repo->objects.objectdir, o->object_dir,\n \t\t\trepo->commondir, \"objects\");\n+\tfree(repo->objects.alternate_db);\n+\trepo->objects.alternate_db = xstrdup_or_null(o->alternate_db);\n \texpand_base_dir(&repo->graft_file, o->graft_file,\n \t\t\trepo->commondir, \"info/grafts\");\n \texpand_base_dir(&repo->index_file, o->index_file,\ndiff --git a/repository.h b/repository.h\nindex b5b5d138aa..b1da2a6384 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -94,6 +94,7 @@ struct set_gitdir_args {\n \tconst char *object_dir;\n \tconst char *graft_file;\n \tconst char *index_file;\n+\tconst char *alternate_db;\n };\n \n extern void repo_set_gitdir(struct repository *repo,\ndiff --git a/sha1_file.c b/sha1_file.c\nindex dfc8deec38..ad1cd441e6 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -673,15 +673,11 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \n void prepare_alt_odb(struct repository *r)\n {\n-\tconst char *alt;\n-\n \tif (r->objects.alt_odb_tail)\n \t\treturn;\n \n-\talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n-\n \tr->objects.alt_odb_tail = &r->objects.alt_odb_list;\n-\tlink_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);\n+\tlink_alt_odb_entries(r, r->objects.alternate_db, PATH_SEP, NULL, 0);\n \n \tread_info_alternates(r, r->objects.objectdir, 0);\n }\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340286","messageId":"20180226103030.26900-5-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180226103030.26900-1-pclouds@gmail.com","subject":"[PATCH 4/4] repository: delete ignore_env member","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-26T10:30:30Z","receivedAt":"2018-02-26T10:31:18Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This variable was added because the repo_set_gitdir() was created to\ncover both submodule and main repos, but these two are initialized a\nbit differently so ignore_env == 0 means main repo, while ignore_env\n!= 0 is submodules.\n\nSince the difference part (env variables) has been moved out of\nrepo_set_gitdir(), this function works the same way for both repo\ntypes and ignore_env is not needed anymore.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n repository.c | 4 +---\n repository.h | 9 ---------\n 2 files changed, 1 insertion(+), 12 deletions(-)\n\ndiff --git a/repository.c b/repository.c\nindex 19532ec5ff..8f6386022f 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -12,7 +12,7 @@ static struct repository the_repo = {\n \tNULL, NULL, NULL,\n \t&the_index,\n \t&hash_algos[GIT_HASH_SHA1],\n-\t0, 0\n+\t0\n };\n struct repository *the_repository = &the_repo;\n \n@@ -134,8 +134,6 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)\n \tstruct repository_format format;\n \tmemset(repo, 0, sizeof(*repo));\n \n-\trepo->ignore_env = 1;\n-\n \tINIT_LIST_HEAD(&repo->objects.packed_git_mru);\n \n \tif (repo_init_gitdir(repo, gitdir))\ndiff --git a/repository.h b/repository.h\nindex b1da2a6384..07e8971428 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -73,15 +73,6 @@ struct repository {\n \tconst struct git_hash_algo *hash_algo;\n \n \t/* Configurations */\n-\t/*\n-\t * Bit used during initialization to indicate if repository state (like\n-\t * the location of the 'objectdir') should be read from the\n-\t * environment. By default this bit will be set at the begining of\n-\t * 'repo_init()' so that all repositories will ignore the environment.\n-\t * The exception to this is 'the_repository', which doesn't go through\n-\t * the normal 'repo_init()' process.\n-\t */\n-\tunsigned ignore_env:1;\n \n \t/* Indicate if a repository has a different 'commondir' from 'gitdir' */\n \tunsigned different_commondir:1;\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340314","messageId":"xmqqsh9nr5dx.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"20180226103030.26900-1-pclouds@gmail.com","subject":"Re: [PATCH 0/4] Delete ignore_env member in struct repository","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-26T18:07:54Z","receivedAt":"2018-02-26T18:08:05Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyễn Thái Ngọc Duy writes:\n\n> This series is built on top of Stefan's object-store-part1, v4. I\n> could rebase it on 'master' too, but then Junio may need to resolve\n> some conflicts.\n\nAs a follow-up fix for combined \"move things to the_repo as much as\npossible\" efforts I think what you did makes most sense. I would\nsay that it makes even more sense to make these part of the\nobject-store series if/when the series needs rerolling--after all\nthe topic is already a multi-author effort.\n\nI'll have to read them carefully before commenting on the actual\npatches ;-) Thanks.\n\n>\n> [1] https://public-inbox.org/git/20180225111840.16421-1-pclouds@gmail.com/\n>\n> Nguyễn Thái Ngọc Duy (4):\n> repository.c: move env-related setup code back to environment.c\n> repository.c: delete dead functions\n> sha1_file.c: move delayed getenv(altdb) back to setup_git_env()\n> repository: delete ignore_env member\n>\n> cache.h | 2 +-\n> environment.c | 13 +++++++--\n> object-store.h | 5 +++-\n> object.c | 1 +\n> repository.c | 79 ++++++++++++++++++++++----------------------------\n> repository.h | 21 +++++++-------\n> setup.c | 3 +-\n> sha1_file.c | 6 +---\n> 8 files changed, 64 insertions(+), 66 deletions(-)\n"},{"id":"340315","messageId":"20180226101926.0ceafa4d7ef611ce142593fe@google.com","threadId":"47823","inReplyTo":"20180224004754.129721-1-sbeller@google.com","subject":"Re: [PATCHv4 00/27] Moving global state into the repository object (part 1)","fromName":"Jonathan Tan","fromEmail":"jonathantanmy@google.com","sentAt":"2018-02-26T18:19:26Z","receivedAt":"2018-02-26T18:19:34Z","isPatch":false,"sender":{"key":"jonathantanmy@fastmail.com","avatar":null},"body":"On Fri, 23 Feb 2018 16:47:27 -0800\nStefan Beller wrote:\n\n> v4:\n> * addressed feedback from the single patches (mostly nits)\n> * rebased on latest master\n\nThe patches I looked at previously (patches 7, 15, 19, 22, and 24) look\ngood to me.\n"},{"id":"340316","messageId":"xmqqo9kbr4u7.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"20180226093040.GA10479@ash","subject":"Re: [PATCHv4 01/27] repository: introduce raw object store field","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-26T18:19:44Z","receivedAt":"2018-02-26T18:19:58Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Duy Nguyen writes:\n\n> diff --git a/common-main.c b/common-main.c\n> index 6a689007e7..a13ab981aa 100644\n> --- a/common-main.c\n> +++ b/common-main.c\n> @@ -1,6 +1,7 @@\n> #include \"cache.h\"\n> #include \"exec_cmd.h\"\n> #include \"attr.h\"\n> +#include \"repository.h\"\n> \n> /*\n> * Many parts of Git have subprograms communicate via pipe, expect the\n> @@ -32,6 +33,8 @@ int main(int argc, const char **argv)\n> \t */\n> \tsanitize_stdfds();\n> \n> +\tinit_the_repository();\n> +\n> \tgit_setup_gettext();\n> ...\n> +void init_the_repository(void)\n> +{\n> +\tthe_repository = &the_repo;\n> +\trepo_pre_init(the_repository);\n> +\tthe_repository->index = &the_index;\n> +\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n> +}\n\nI see what you did here, and I like it.\n"},{"id":"340317","messageId":"20180226182324.GA60857@google.com","threadId":"47823","inReplyTo":"xmqqo9kbr4u7.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCHv4 01/27] repository: introduce raw object store field","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-26T18:23:24Z","receivedAt":"2018-02-26T18:23:31Z","isPatch":false,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/26, Junio C Hamano wrote:\n> Duy Nguyen writes:\n> \n> > diff --git a/common-main.c b/common-main.c\n> > index 6a689007e7..a13ab981aa 100644\n> > --- a/common-main.c\n> > +++ b/common-main.c\n> > @@ -1,6 +1,7 @@\n> > #include \"cache.h\"\n> > #include \"exec_cmd.h\"\n> > #include \"attr.h\"\n> > +#include \"repository.h\"\n> > \n> > /*\n> > * Many parts of Git have subprograms communicate via pipe, expect the\n> > @@ -32,6 +33,8 @@ int main(int argc, const char **argv)\n> > \t */\n> > \tsanitize_stdfds();\n> > \n> > +\tinit_the_repository();\n> > +\n> > \tgit_setup_gettext();\n> > ...\n> > +void init_the_repository(void)\n> > +{\n> > +\tthe_repository = &the_repo;\n> > +\trepo_pre_init(the_repository);\n> > +\tthe_repository->index = &the_index;\n> > +\trepo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n> > +}\n> \n> I see what you did here, and I like it.\n\nI thought this would be a good idea to do eventually but back when I\nfirst introduced struct repository there wasn't enough to justify it\ntill now. This definitely makes it much easier to read the\ninitialization and I prefer this over the initializer. Thanks for\nworking on this :)\n\n-- \nBrandon Williams\n"},{"id":"340319","messageId":"CAPig+cRkEzN-dHh9R0cX5gOhsJ7X5evAMfpWuCnVp_QN3uz8ng@mail.gmail.com","threadId":"47823","inReplyTo":"20180226103030.26900-2-pclouds@gmail.com","subject":"Re: [PATCH 1/4] repository.c: move env-related setup code back to environment.c","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-02-26T18:57:21Z","receivedAt":"2018-02-26T18:57:29Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Feb 26, 2018 at 5:30 AM, Nguyễn Thái Ngọc Duy wrote:\n> It does not make sense that generic repository code contains handling\n> of environment variables, which are specific for the main repository\n> only. Refactor repo_set_gitdir() function to take $GIT_DIR and\n> optionally _all_ other customizable paths. These optional paths can be\n> NULL and will be calculated according to the default directory layout.\n>\n> Note that some dead functions are left behind to reduce diff\n> noise. They will be deleted in the next patch.\n>\n> Signed-off-by: Nguyễn Thái Ngọc Duy \n> ---\n> diff --git a/environment.c b/environment.c\n> @@ -148,10 +148,17 @@ static char *expand_namespace(const char *raw_namespace)\n> -void setup_git_env(void)\n> +void setup_git_env(const char *git_dir)\n> {\n> const char *shallow_file;\n> const char *replace_ref_base;\n> + struct set_gitdir_args args = { NULL };\n> +\n> + args.shared_root = getenv(GIT_COMMON_DIR_ENVIRONMENT);\n> + args.object_dir = getenv(DB_ENVIRONMENT);\n> + args.graft_file = getenv(GRAFT_ENVIRONMENT);\n> + args.index_file = getenv(INDEX_ENVIRONMENT);\n\nAccording to POSIX[1], the result of getenv() may be invalidated by\nanother call to getenv() (or setenv() or unsetenv() or putenv()).\n\n[1]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/getenv.html\n\n> + repo_set_gitdir(the_repository, git_dir, &args);\n> diff --git a/repository.c b/repository.c\n> @@ -61,15 +61,50 @@ static void repo_setup_env(struct repository *repo)\n> +void repo_set_gitdir(struct repository *repo,\n> + const char *root,\n> + const struct set_gitdir_args *o)\n> +{\n> + const char *gitfile = read_gitfile(root);\n> + char *old_gitdir = repo->gitdir;\n> +\n> + repo->gitdir = xstrdup(gitfile ? gitfile : root);\n> free(old_gitdir);\n\nCan:\n\n char *old_gitdir = repo->gitdir;\n repo->gitdir = xstrdup(...);\n free(old_gitdir);\n\nbe simplified to:\n\n free(repo->gitdir);\n repo->gitdir = xstrdup(...);\n\n?\n\n> + repo_set_commondir(repo, o->shared_root);\n\nThe repo_set_gitdir() prototype (below) makes it seem as if the last\nargument ('o', in this case) is optional, presumably by passing in\nNULL, however, this code does not appear to be prepared to deal with\nNULL.\n\n> + expand_base_dir(&repo->objects.objectdir, o->object_dir,\n> + repo->commondir, \"objects\");\n> + expand_base_dir(&repo->graft_file, o->graft_file,\n> + repo->commondir, \"info/grafts\");\n> + expand_base_dir(&repo->index_file, o->index_file,\n> + repo->gitdir, \"index\");\n> }\n> diff --git a/repository.h b/repository.h\n> @@ -89,7 +89,16 @@ struct repository {\n> +struct set_gitdir_args {\n> + [...]\n> +};\n> +\n> +extern void repo_set_gitdir(struct repository *repo,\n> + const char *root,\n> + const struct set_gitdir_args *optional);\n"},{"id":"340320","messageId":"CAGZ79kY-n-4xnN450+U6_dn2u27ZZE5fKCY4sKKA_yGE=gUOMA@mail.gmail.com","threadId":"47823","inReplyTo":"20180225203401.GF4620@genre.crustytoothpaste.net","subject":"Re: [PATCH v3 0/6] Fix initializing the_hash_algo","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-26T19:20:51Z","receivedAt":"2018-02-26T19:20:58Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Sun, Feb 25, 2018 at 12:34 PM, brian m. carlson\n wrote:\n> On Sun, Feb 25, 2018 at 06:18:34PM +0700, Nguyễn Thái Ngọc Duy wrote:\n>> v3 refines v2 a bit more:\n>>\n>> - fix configure typo (and stray words in commit message)\n>> - use repo_set_hash_algo() instead of reassigning the_hash_algo\n>> - compare hash algos by format_id\n>> - catch NULL hash algo, report nicely and suggest GIT_HASH_FIXUP\n>>\n>> The last point makes me much happier about keeping this workaround\n>> around until we are confident we can live without it. Interdiff\n>\n> This looks sane to me.\n>\n> Reviewed-by: brian m. carlson \n\nI agree with this version as well.\n\nThanks,\nStefan\n"},{"id":"340321","messageId":"CAGZ79kZDhoWWPpoENwvE6esxzdJvTnL4EAxGX5HV=DwDtDOEgw@mail.gmail.com","threadId":"47823","inReplyTo":"20180226093040.GA10479@ash","subject":"Re: [PATCHv4 01/27] repository: introduce raw object store field","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-26T19:28:02Z","receivedAt":"2018-02-26T19:28:09Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Feb 26, 2018 at 1:30 AM, Duy Nguyen wrote:\n> On Fri, Feb 23, 2018 at 04:47:28PM -0800, Stefan Beller wrote:\n>> /* The main repository */\n>> static struct repository the_repo = {\n>> - NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0\n>> + NULL, NULL,\n>> + RAW_OBJECT_STORE_INIT,\n>> + NULL, NULL, NULL,\n>> + NULL, NULL, NULL,\n>> + &the_index,\n>> + &hash_algos[GIT_HASH_SHA1],\n>> + 0, 0\n>> };\n>> struct repository *the_repository = &the_repo;\n>\n> I wonder if we should do something like this. It makes the_repo\n> initialization easier to read and it helps unify the init code with\n> submodule.\n>\n> No don't reroll. If you think it's a good idea, you can do something\n> like this in the next series instead.\n>\n> -- 8< --\n> diff --git a/check-racy.c b/check-racy.c\n> index 24b6542352..47cbb4eb6d 100644\n> --- a/check-racy.c\n> +++ b/check-racy.c\n\ntotally offtopic: Do we want to move this file into t/helper?\n\n> --- a/common-main.c\n> +++ b/common-main.c\n> @@ -1,6 +1,7 @@\n> #include \"cache.h\"\n> #include \"exec_cmd.h\"\n> #include \"attr.h\"\n> +#include \"repository.h\"\n>\n> /*\n> * Many parts of Git have subprograms communicate via pipe, expect the\n> @@ -32,6 +33,8 @@ int main(int argc, const char **argv)\n> */\n> sanitize_stdfds();\n>\n> + init_the_repository();\n> +\n\nSo this is the real deal.\n\n> -#define RAW_OBJECT_STORE_INIT { NULL }\n>\n> +void raw_object_store_init(struct raw_object_store *o);\n> void raw_object_store_clear(struct raw_object_store *o);\n>\n> #endif /* OBJECT_STORE_H */\n> diff --git a/object.c b/object.c\n> index 11d904c033..8a4d01dd5f 100644\n> --- a/object.c\n> +++ b/object.c\n> @@ -446,6 +446,11 @@ void clear_commit_marks_all(unsigned int flags)\n> }\n> }\n>\n> +void raw_object_store_init(struct raw_object_store *o)\n> +{\n> + memset(o, 0, sizeof(*o));\n> +}\n\nWe'll have to adapt this for the list that we use for packed_git_mru,\nbut that should be no problem.\n\n> +\n> +static void repo_pre_init(struct repository *repo)\n> +{\n> + memset(repo, 0, sizeof(*repo));\n> + raw_object_store_init(&repo->objects);\n> +}\n> +\n> +void init_the_repository(void)\n> +{\n> + the_repository = &the_repo;\n> + repo_pre_init(the_repository);\n> + the_repository->index = &the_index;\n> + repo_set_hash_algo(the_repository, GIT_HASH_SHA1);\n> +}\n>\n> static char *git_path_from_env(const char *envvar, const char *git_dir,\n> const char *path, int fromenv)\n> @@ -138,7 +144,8 @@ static int read_and_verify_repository_format(struct repository_format *format,\n> int repo_init(struct repository *repo, const char *gitdir, const char *worktree)\n> {\n> struct repository_format format;\n> - memset(repo, 0, sizeof(*repo));\n> +\n> + repo_pre_init(repo);\n>\n> repo->ignore_env = 1;\n>\n\nI like the approach. Though as you say, let's have it in the next series.\n\nThanks for the thoughts and guidance,\nStefan\n"},{"id":"340322","messageId":"CAGZ79kZHw3qr+-0QeE3j0nSjMkob8-QOCcbHRfoZGbapQsURaw@mail.gmail.com","threadId":"47823","inReplyTo":"20180226103030.26900-2-pclouds@gmail.com","subject":"Re: [PATCH 1/4] repository.c: move env-related setup code back to environment.c","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-26T20:30:57Z","receivedAt":"2018-02-26T20:31:04Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"> diff --git a/setup.c b/setup.c\n> index c5d55dcee4..6fac1bb58a 100644\n> --- a/setup.c\n> +++ b/setup.c\n> @@ -1116,8 +1116,7 @@ const char *setup_git_directory_gently(int *nongit_ok)\n> const char *gitdir = getenv(GIT_DIR_ENVIRONMENT);\n> if (!gitdir)\n> gitdir = DEFAULT_GIT_DIR_ENVIRONMENT;\n> - repo_set_gitdir(the_repository, gitdir);\n> - setup_git_env();\n> + setup_git_env(gitdir);\n> }\n\nWhat makes git_dir special, such that we have to pass it in here?\nCould the check for getenv(GIT_DIR_ENVIRONMENT) and fallback to\nDEFAULT_... be part of setup_git_env() ?\nOh I guess that cannot be done easily as the submodule code\nspcifically doesn't want to discover the git dir itself.\n\n\n> if (startup_info->have_repository)\n> repo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n> --\n> 2.16.1.435.g8f24da2e1a\n>\n"},{"id":"340324","messageId":"CAGZ79kbyrOc6zDSC4cirUW4ZLbDtEeOB=JwPrqxyye=wFR197Q@mail.gmail.com","threadId":"47823","inReplyTo":"20180226103030.26900-1-pclouds@gmail.com","subject":"Re: [PATCH 0/4] Delete ignore_env member in struct repository","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-26T20:46:11Z","receivedAt":"2018-02-26T20:46:21Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Feb 26, 2018 at 2:30 AM, Nguyễn Thái Ngọc Duy wrote:\n> It turns out I don't need my other series [1] in order to delete this\n> field. This series moves getenv() calls from\n> repo_set_gitdir()/repo_setup_env() and prepare_alt_odb() back in\n> environment.c where they belong in my opinion.\n>\n> The repo_set_gitdir() now takes $GIT_DIR and optionally all other\n> configurable paths. If those paths are NULL, default repo layout will\n> be used. With getenv() no longer called inside repo_set_gitdir(),\n> ignore_env has no reason to stay. This is in 1/4.\n>\n> The getenv() in prepare_alt_odb() is also moved back to\n> setup_git_env() in 3/4. It demonstrates how we could move other\n> getenv() back to if we want.\n>\n> This series is built on top of Stefan's object-store-part1, v4. I\n> could rebase it on 'master' too, but then Junio may need to resolve\n> some conflicts.\n>\n\nThanks for working on this,\nI found this series a pleasant read, the only issue I saw was Erics upbringing\nof multiple getenv calls without strdup()ing the content.\n\nWhat is the plan from here on?\nShould I build further series on top of yours? The next series will\nfocus on the pack side of things (pack.h, packfile.{c,h})\n\nSo maybe we'll have Junio merge down my series (and yours as it\nneeds one reroll?) and then build on top of that?\n\nThanks,\nStefan\n"},{"id":"340327","messageId":"CAGZ79kbgYwCjDT3wviFXakqhAKf51AvTyJOPDCW_6OtxGax87g@mail.gmail.com","threadId":"47823","inReplyTo":"CACsJy8AS5RiuXeM2u34i0hjKXJqEhsrb4Ysi9Md9OhRpP4_B-A@mail.gmail.com","subject":"Re: [PATCHv4 00/27] Moving global state into the repository object (part 1)","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-26T20:50:42Z","receivedAt":"2018-02-26T20:50:51Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Sat, Feb 24, 2018 at 7:00 AM, Duy Nguyen wrote:\n> I notice that there are a few global state (or configuration rather)\n> left after this: packed_git_window_size, packed_git_limit and\n> delta_base_cache_limit. These can be changed by $GIT_DIR/config, but\n> since it's still global, a submodule repository will use the same\n> settings of its super repository. If $SUBMODULE/config changes any of\n> these, they are ignored.\n\nThat sounds all packing related, which I plan on working on next.\n\n> The natural thing to do is move these to raw_object_store too (and\n> repo_submodule_init needs to check submodule's config file). But one\n> may argue these should be per-process instead of per-repo though. I\n> don't know. But I thought I should mention this.\n\nFor now a process and a repository is the same as git-gc or git-repack\ndoesn't know about the --recurse-submodules flag, yet.\nI wonder if we ever want to teach those commands the submodule\nrecursion, because of the issue you bring up, which settings do we apply\nfor a submodule? Traditionally we'd just have the command line override\nthe configuration, which I don't know if it is a good idea for these settings.\n\nThanks,\nStefan\n"},{"id":"340328","messageId":"CAGZ79kbuPUb6iHJo8B5YzB12oV79ujCR=zaSE=61xrKBRcuHjg@mail.gmail.com","threadId":"47823","inReplyTo":"20180226085508.GA30343@sigill.intra.peff.net","subject":"Re: [PATCH 08/27] pack: move approximate object count to object store","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-26T20:57:15Z","receivedAt":"2018-02-26T20:57:23Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Mon, Feb 26, 2018 at 12:55 AM, Jeff King wrote:\n> On Fri, Feb 23, 2018 at 02:22:14PM -0800, Stefan Beller wrote:\n>\n>> >> + /*\n>> >> + * A fast, rough count of the number of objects in the repository.\n>> >> + * These two fields are not meant for direct access. Use\n>> >> + * approximate_object_count() instead.\n>> >> + */\n>> >> + unsigned long approximate_object_count;\n>> >> + unsigned approximate_object_count_valid : 1;\n>> >\n>> > Patch looks fine and is effectively a no-op, though what is the need for\n>> > both of these variables? Maybe it can be simplified down to just use\n>> > one? Just musing as its out of the scope of this patch and we probably\n>> > shouldn't try to change that in this series.\n>>\n>> I agree we should. It was introduced in e323de4ad7f (sha1_file:\n>> allow sha1_loose_object_info to handle arbitrary repositories, 2017-08-30)\n>> and I think it was seen as a clever optimization trick back then?\n>\n> I think you meant 8e3f52d778 (find_unique_abbrev: move logic out of\n> get_short_sha1(), 2016-10-03)?\n\nYes, copy paste error.\n\n>\n> Yes, it was just to avoid the dual-meaning of \"0\" for \"not set\" and a\n> repository with no packfiles. It would probably be fine to get rid of\n> it. If you have no packfiles then you probably don't have enough objects\n> to worry about micro-optimizing. And anyway, the \"wasted\" case wouldn't\n> even make any syscalls (it would do a noop prepare_packed_git() and then\n> realize the packed_git list is empty).\n\nGood point!\n\n>\n> -Peff\n"},{"id":"340345","messageId":"xmqqfu5nqrf8.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"CACsJy8AUvAv61wM5hnvhyCaOd_oKmMbhAyDGJ9pmkiSu_WveZQ@mail.gmail.com","subject":"Re: [PATCH 2/2] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-26T23:09:31Z","receivedAt":"2018-02-26T23:09:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Duy Nguyen writes:\n\n> Yes that's the intention. But after writing cover letter for v2 and\n> sending it out, it looks to me that this thing must stay until all our\n> code is converted to using the_hash_algo (I don't know if there are\n> more to convert or it's finished already). So an alternative is we do\n> the opposite: default to GIT_HASH_SHA1, but when an env variable is\n> set, reset it back to NULL. This env variable will _always_ be set by\n> the test suite to help us catch problems.\n\nIf we were to do the \"do not blindly initialize and always\ninitialize explicitly for debugging\" (which I doubt is a good\ndirection to go in), then what you outlined above certainly is a\nless evil way to do so than what the patch under discussion did, I\nwould think.\n"},{"id":"340350","messageId":"CACsJy8Csr+u9S5hs=jyEqpb3=4zpR8Ap9ks29jV4YFO1eSR5vQ@mail.gmail.com","threadId":"47823","inReplyTo":"CAGZ79kZDhoWWPpoENwvE6esxzdJvTnL4EAxGX5HV=DwDtDOEgw@mail.gmail.com","subject":"Re: [PATCHv4 01/27] repository: introduce raw object store field","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-26T23:59:33Z","receivedAt":"2018-02-27T00:00:10Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Feb 27, 2018 at 2:28 AM, Stefan Beller wrote:\n> On Mon, Feb 26, 2018 at 1:30 AM, Duy Nguyen wrote:\n>> On Fri, Feb 23, 2018 at 04:47:28PM -0800, Stefan Beller wrote:\n>>> /* The main repository */\n>>> static struct repository the_repo = {\n>>> - NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &the_index, &hash_algos[GIT_HASH_SHA1], 0, 0\n>>> + NULL, NULL,\n>>> + RAW_OBJECT_STORE_INIT,\n>>> + NULL, NULL, NULL,\n>>> + NULL, NULL, NULL,\n>>> + &the_index,\n>>> + &hash_algos[GIT_HASH_SHA1],\n>>> + 0, 0\n>>> };\n>>> struct repository *the_repository = &the_repo;\n>>\n>> I wonder if we should do something like this. It makes the_repo\n>> initialization easier to read and it helps unify the init code with\n>> submodule.\n>>\n>> No don't reroll. If you think it's a good idea, you can do something\n>> like this in the next series instead.\n>>\n>> -- 8< --\n>> diff --git a/check-racy.c b/check-racy.c\n>> index 24b6542352..47cbb4eb6d 100644\n>> --- a/check-racy.c\n>> +++ b/check-racy.c\n>\n> totally offtopic: Do we want to move this file into t/helper?\n\nNo wonder both Jeff and I missed this program (he didn't convert it to\nuse cmd_main, and I didn't move it to t/helper). This git-check-racy\nis added in 42f774063d (Add check program \"git-check-racy\" -\n2006-08-15) and is not part of the default build. You need to manually\nupdate Makefile first to build it.\n\nRight now it's broken (multiple definition of 'main'). If you add\ninit_the_repository() or something similar, just leave this file\nuntouched. Maybe I'll fix it later, separately. Or perhaps I'll move\nthis functionality to git-update-index if this is still worth keeping.\n-- \nDuy\n"},{"id":"340352","messageId":"CACsJy8BDxbvAigg2wVh82jh3iQ1gy5dnCh0zXQ64B6QqrGE0tA@mail.gmail.com","threadId":"47823","inReplyTo":"CAGZ79kbgYwCjDT3wviFXakqhAKf51AvTyJOPDCW_6OtxGax87g@mail.gmail.com","subject":"Re: [PATCHv4 00/27] Moving global state into the repository object (part 1)","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-27T00:02:40Z","receivedAt":"2018-02-27T00:03:15Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Feb 27, 2018 at 3:50 AM, Stefan Beller wrote:\n>> The natural thing to do is move these to raw_object_store too (and\n>> repo_submodule_init needs to check submodule's config file). But one\n>> may argue these should be per-process instead of per-repo though. I\n>> don't know. But I thought I should mention this.\n>\n> For now a process and a repository is the same as git-gc or git-repack\n\nI think you're thinking about the pack writing part, but these are\nconfiguration for pack reading (aka \"object store\"). If you read a\nblob from a submodule (e.g. git-grep), you'll use these configurations\nat some point.\n\nThere are of course more configuration for pack writing (like\nzlib_compression_level) which I deliberately avoided to mention\nbecause I don't even know where they belong.\n\n> doesn't know about the --recurse-submodules flag, yet.\n> I wonder if we ever want to teach those commands the submodule\n> recursion, because of the issue you bring up, which settings do we apply\n> for a submodule? Traditionally we'd just have the command line override\n> the configuration, which I don't know if it is a good idea for these settings.\n-- \nDuy\n"},{"id":"340354","messageId":"CACsJy8AK_q1FK7mzAUOMAy3324YrMrTD3oRaAomDXetFLDmiXQ@mail.gmail.com","threadId":"47823","inReplyTo":"CAGZ79kbyrOc6zDSC4cirUW4ZLbDtEeOB=JwPrqxyye=wFR197Q@mail.gmail.com","subject":"Re: [PATCH 0/4] Delete ignore_env member in struct repository","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-27T00:18:58Z","receivedAt":"2018-02-27T00:19:34Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Feb 27, 2018 at 3:46 AM, Stefan Beller wrote:\n> What is the plan from here on?\n> Should I build further series on top of yours? The next series will\n> focus on the pack side of things (pack.h, packfile.{c,h})\n>\n> So maybe we'll have Junio merge down my series (and yours as it\n> needs one reroll?) and then build on top of that?\n\nI think that's the less painful way for everybody.\n-- \nDuy\n"},{"id":"340355","messageId":"CACsJy8Bpo9yPEuiHe1+Rz4JEuT=8kkhyW3d-6_T2UbnoaorE5g@mail.gmail.com","threadId":"47823","inReplyTo":"CAGZ79kZHw3qr+-0QeE3j0nSjMkob8-QOCcbHRfoZGbapQsURaw@mail.gmail.com","subject":"Re: [PATCH 1/4] repository.c: move env-related setup code back to environment.c","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-27T00:28:38Z","receivedAt":"2018-02-27T00:29:14Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Feb 27, 2018 at 3:30 AM, Stefan Beller wrote:\n>> diff --git a/setup.c b/setup.c\n>> index c5d55dcee4..6fac1bb58a 100644\n>> --- a/setup.c\n>> +++ b/setup.c\n>> @@ -1116,8 +1116,7 @@ const char *setup_git_directory_gently(int *nongit_ok)\n>> const char *gitdir = getenv(GIT_DIR_ENVIRONMENT);\n>> if (!gitdir)\n>> gitdir = DEFAULT_GIT_DIR_ENVIRONMENT;\n>> - repo_set_gitdir(the_repository, gitdir);\n>> - setup_git_env();\n>> + setup_git_env(gitdir);\n>> }\n>\n> What makes git_dir special, such that we have to pass it in here?\n> Could the check for getenv(GIT_DIR_ENVIRONMENT) and fallback to\n> DEFAULT_... be part of setup_git_env() ?\n> Oh I guess that cannot be done easily as the submodule code\n> spcifically doesn't want to discover the git dir itself.\n\nNo, submodule code already does not care about setup_git_env().\n\nBack to your first question, this is related to the \"NEEDSWORK\"\ncomment in this code. We _should_ always set_git_dir() before leaving\nsetup_git_directory_gently() then everybody behaves the same way and\nwe don't need this special setup_git_env() here at all. BUT we need to\nbe careful about setting environment variable $GIT_DIR, done inside\nset_git_dir(), because sub-processes will inherit it and if we're not\ncareful, we'll break .git discovery in those. There's another thing I\ndon't like about this is setup_work_tree() using set_git_dir() the\nsecond time to adjust relative $GIT_DIR and friends when it moves\n$CWD. I'll need to fix this, soon.\n\nSo in short, it's special because the current situation is messy and\n(hopefully) temporary. But it should eventually be gone.\n-- \nDuy\n"},{"id":"340388","messageId":"CACsJy8AqXNGMdCmF5smooLyWBuiKmXG6mckquiPekWZ8GwF5Qg@mail.gmail.com","threadId":"47823","inReplyTo":"CAPig+cRkEzN-dHh9R0cX5gOhsJ7X5evAMfpWuCnVp_QN3uz8ng@mail.gmail.com","subject":"Re: [PATCH 1/4] repository.c: move env-related setup code back to environment.c","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-27T09:11:44Z","receivedAt":"2018-02-27T09:12:22Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Feb 27, 2018 at 1:57 AM, Eric Sunshine wrote:\n\n>> + repo_set_commondir(repo, o->shared_root);\n>\n> The repo_set_gitdir() prototype (below) makes it seem as if the last\n> argument ('o', in this case) is optional, presumably by passing in\n> NULL, however, this code does not appear to be prepared to deal with\n> NULL.\n\nI know. I went this this struct because I really hate to write lots of\nNULL without names\n\n repo_set_gitdir(gitdir, NULL, NULL, NULL, ...);\n\nbut I could not find a nicer way to make the whole struct optional,\nnot without repeating more code or using preprocessor. Given that this\nfunction has 3-4 call sites max, I think it's still ok.\n\n>\n>> + expand_base_dir(&repo->objects.objectdir, o->object_dir,\n>> + repo->commondir, \"objects\");\n>> + expand_base_dir(&repo->graft_file, o->graft_file,\n>> + repo->commondir, \"info/grafts\");\n>> + expand_base_dir(&repo->index_file, o->index_file,\n>> + repo->gitdir, \"index\");\n>> }\n-- \nDuy\n"},{"id":"340389","messageId":"CACsJy8BAsQPwgBMfK4BHB-GiKoknphuTT8n=QizU0rT6K5TB2Q@mail.gmail.com","threadId":"47823","inReplyTo":"CAPig+cRkEzN-dHh9R0cX5gOhsJ7X5evAMfpWuCnVp_QN3uz8ng@mail.gmail.com","subject":"Re: [PATCH 1/4] repository.c: move env-related setup code back to environment.c","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-27T09:39:25Z","receivedAt":"2018-02-27T09:40:04Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Feb 27, 2018 at 1:57 AM, Eric Sunshine wrote:\n> Can:\n>\n> char *old_gitdir = repo->gitdir;\n> repo->gitdir = xstrdup(...);\n> free(old_gitdir);\n>\n> be simplified to:\n>\n> free(repo->gitdir);\n> repo->gitdir = xstrdup(...);\n>\n> ?\n\nNo because I think \"root\" (not quoted here) could point to the same\nvalue as repo->gitdir, if you free that first, the second xstrdup()\nwill duplicate a freed memory. See 1fb2b636c6 (set_git_dir: handle\nfeeding gitdir to itself - 2017-09-05)\n\nI will add a comment about this since it's not very clear from this code.\n-- \nDuy\n"},{"id":"340390","messageId":"20180227095846.9238-1-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180226103030.26900-1-pclouds@gmail.com","subject":"[PATCH v2 0/4] Delete ignore_env member in struct repository","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-27T09:58:42Z","receivedAt":"2018-02-27T09:59:00Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"v2 fixes the incorrect use of consecutive getenv() and adds a comment\nto clarify the role of old_gitdir\n\nInterdiff:\n\ndiff --git a/environment.c b/environment.c\nindex 95de419de8..47c6e31559 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -14,6 +14,7 @@\n #include \"fmt-merge-msg.h\"\n #include \"commit.h\"\n #include \"object-store.h\"\n+#include \"argv-array.h\"\n \n int trust_executable_bit = 1;\n int trust_ctime = 1;\n@@ -148,18 +149,34 @@ static char *expand_namespace(const char *raw_namespace)\n \treturn strbuf_detach(&buf, NULL);\n }\n \n+/* Wrapper of getenv() that returns a strdup value. This value is kept\n+ * in argv to be freed later.\n+ */\n+static const char *getenv_safe(struct argv_array *argv, const char *name)\n+{\n+\tconst char *value = getenv(name);\n+\n+\tif (!value)\n+\t\treturn NULL;\n+\n+\targv_array_push(argv, value);\n+\treturn argv->argv[argv->argc - 1];\n+}\n+\n void setup_git_env(const char *git_dir)\n {\n \tconst char *shallow_file;\n \tconst char *replace_ref_base;\n \tstruct set_gitdir_args args = { NULL };\n+\tstruct argv_array to_free = ARGV_ARRAY_INIT;\n \n-\targs.shared_root = getenv(GIT_COMMON_DIR_ENVIRONMENT);\n-\targs.object_dir = getenv(DB_ENVIRONMENT);\n-\targs.graft_file = getenv(GRAFT_ENVIRONMENT);\n-\targs.index_file = getenv(INDEX_ENVIRONMENT);\n-\targs.alternate_db = getenv(ALTERNATE_DB_ENVIRONMENT);\n+\targs.shared_root = getenv_safe(&to_free, GIT_COMMON_DIR_ENVIRONMENT);\n+\targs.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);\n+\targs.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);\n+\targs.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);\n+\targs.alternate_db = getenv_safe(&to_free, ALTERNATE_DB_ENVIRONMENT);\n \trepo_set_gitdir(the_repository, git_dir, &args);\n+\targv_array_clear(&to_free);\n \n \tif (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))\n \t\tcheck_replace_refs = 0;\ndiff --git a/repository.c b/repository.c\nindex 8f6386022f..c555dacad2 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -48,6 +48,11 @@ void repo_set_gitdir(struct repository *repo,\n \t\t const struct set_gitdir_args *o)\n {\n \tconst char *gitfile = read_gitfile(root);\n+\t/*\n+\t * repo->gitdir is saved because the caller could pass \"root\"\n+\t * that also points to repo->gitdir. We want to keep it alive\n+\t * until after xstrdup(root). Then we can free it.\n+\t */\n \tchar *old_gitdir = repo->gitdir;\n \n \trepo->gitdir = xstrdup(gitfile ? gitfile : root);\n\nNguyễn Thái Ngọc Duy (4):\n repository.c: move env-related setup code back to environment.c\n repository.c: delete dead functions\n sha1_file.c: move delayed getenv(altdb) back to setup_git_env()\n repository: delete ignore_env member\n\n cache.h | 2 +-\n environment.c | 30 ++++++++++++++++--\n object-store.h | 5 ++-\n object.c | 1 +\n repository.c | 84 ++++++++++++++++++++++++--------------------------\n repository.h | 21 +++++++------\n setup.c | 3 +-\n sha1_file.c | 6 +---\n 8 files changed, 86 insertions(+), 66 deletions(-)\n\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340391","messageId":"20180227095846.9238-2-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180227095846.9238-1-pclouds@gmail.com","subject":"[PATCH v2 1/4] repository.c: move env-related setup code back to environment.c","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-27T09:58:43Z","receivedAt":"2018-02-27T09:59:05Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"It does not make sense that generic repository code contains handling\nof environment variables, which are specific for the main repository\nonly. Refactor repo_set_gitdir() function to take $GIT_DIR and\noptionally _all_ other customizable paths. These optional paths can be\nNULL and will be calculated according to the default directory layout.\n\nNote that some dead functions are left behind to reduce diff\nnoise. They will be deleted in the next patch.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n cache.h | 2 +-\n environment.c | 29 +++++++++++++++++++++++++---\n repository.c | 53 +++++++++++++++++++++++++++++++++++++++++++++------\n repository.h | 11 ++++++++++-\n setup.c | 3 +--\n 5 files changed, 85 insertions(+), 13 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 5717399183..b164a407eb 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -459,7 +459,7 @@ static inline enum object_type object_type(unsigned int mode)\n */\n extern const char * const local_repo_env[];\n \n-extern void setup_git_env(void);\n+extern void setup_git_env(const char *git_dir);\n \n /*\n * Returns true iff we have a configured git repository (either via\ndiff --git a/environment.c b/environment.c\nindex ec10b062e6..74a2900ddf 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -14,6 +14,7 @@\n #include \"fmt-merge-msg.h\"\n #include \"commit.h\"\n #include \"object-store.h\"\n+#include \"argv-array.h\"\n \n int trust_executable_bit = 1;\n int trust_ctime = 1;\n@@ -148,10 +149,33 @@ static char *expand_namespace(const char *raw_namespace)\n \treturn strbuf_detach(&buf, NULL);\n }\n \n-void setup_git_env(void)\n+/* Wrapper of getenv() that returns a strdup value. This value is kept\n+ * in argv to be freed later.\n+ */\n+static const char *getenv_safe(struct argv_array *argv, const char *name)\n+{\n+\tconst char *value = getenv(name);\n+\n+\tif (!value)\n+\t\treturn NULL;\n+\n+\targv_array_push(argv, value);\n+\treturn argv->argv[argv->argc - 1];\n+}\n+\n+void setup_git_env(const char *git_dir)\n {\n \tconst char *shallow_file;\n \tconst char *replace_ref_base;\n+\tstruct set_gitdir_args args = { NULL };\n+\tstruct argv_array to_free = ARGV_ARRAY_INIT;\n+\n+\targs.shared_root = getenv_safe(&to_free, GIT_COMMON_DIR_ENVIRONMENT);\n+\targs.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);\n+\targs.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);\n+\targs.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);\n+\trepo_set_gitdir(the_repository, git_dir, &args);\n+\targv_array_clear(&to_free);\n \n \tif (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))\n \t\tcheck_replace_refs = 0;\n@@ -301,8 +325,7 @@ int set_git_dir(const char *path)\n {\n \tif (setenv(GIT_DIR_ENVIRONMENT, path, 1))\n \t\treturn error(\"Could not set GIT_DIR to '%s'\", path);\n-\trepo_set_gitdir(the_repository, path);\n-\tsetup_git_env();\n+\tsetup_git_env(path);\n \treturn 0;\n }\n \ndiff --git a/repository.c b/repository.c\nindex a069b1b640..343efe7282 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -61,15 +61,55 @@ static void repo_setup_env(struct repository *repo)\n \t\t\t\t\t \"index\", !repo->ignore_env);\n }\n \n-void repo_set_gitdir(struct repository *repo, const char *path)\n+static void expand_base_dir(char **out, const char *in,\n+\t\t\t const char *base_dir, const char *def_in)\n {\n-\tconst char *gitfile = read_gitfile(path);\n-\tchar *old_gitdir = repo->gitdir;\n+\tfree(*out);\n+\tif (in)\n+\t\t*out = xstrdup(in);\n+\telse\n+\t\t*out = xstrfmt(\"%s/%s\", base_dir, def_in);\n+}\n+\n+static void repo_set_commondir(struct repository *repo,\n+\t\t\t const char *shared_root)\n+{\n+\tstruct strbuf sb = STRBUF_INIT;\n+\n+\tfree(repo->commondir);\n \n-\trepo->gitdir = xstrdup(gitfile ? gitfile : path);\n-\trepo_setup_env(repo);\n+\tif (shared_root) {\n+\t\trepo->different_commondir = 1;\n+\t\trepo->commondir = xstrdup(shared_root);\n+\t\treturn;\n+\t}\n \n+\trepo->different_commondir = get_common_dir_noenv(&sb, repo->gitdir);\n+\trepo->commondir = strbuf_detach(&sb, NULL);\n+}\n+\n+void repo_set_gitdir(struct repository *repo,\n+\t\t const char *root,\n+\t\t const struct set_gitdir_args *o)\n+{\n+\tconst char *gitfile = read_gitfile(root);\n+\t/*\n+\t * repo->gitdir is saved because the caller could pass \"root\"\n+\t * that also points to repo->gitdir. We want to keep it alive\n+\t * until after xstrdup(root). Then we can free it.\n+\t */\n+\tchar *old_gitdir = repo->gitdir;\n+\n+\trepo->gitdir = xstrdup(gitfile ? gitfile : root);\n \tfree(old_gitdir);\n+\n+\trepo_set_commondir(repo, o->shared_root);\n+\texpand_base_dir(&repo->objects.objectdir, o->object_dir,\n+\t\t\trepo->commondir, \"objects\");\n+\texpand_base_dir(&repo->graft_file, o->graft_file,\n+\t\t\trepo->commondir, \"info/grafts\");\n+\texpand_base_dir(&repo->index_file, o->index_file,\n+\t\t\trepo->gitdir, \"index\");\n }\n \n void repo_set_hash_algo(struct repository *repo, int hash_algo)\n@@ -87,6 +127,7 @@ static int repo_init_gitdir(struct repository *repo, const char *gitdir)\n \tint error = 0;\n \tchar *abspath = NULL;\n \tconst char *resolved_gitdir;\n+\tstruct set_gitdir_args args = { NULL };\n \n \tabspath = real_pathdup(gitdir, 0);\n \tif (!abspath) {\n@@ -101,7 +142,7 @@ static int repo_init_gitdir(struct repository *repo, const char *gitdir)\n \t\tgoto out;\n \t}\n \n-\trepo_set_gitdir(repo, resolved_gitdir);\n+\trepo_set_gitdir(repo, resolved_gitdir, &args);\n \n out:\n \tfree(abspath);\ndiff --git a/repository.h b/repository.h\nindex fa73ab8e93..b5b5d138aa 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -89,7 +89,16 @@ struct repository {\n \n extern struct repository *the_repository;\n \n-extern void repo_set_gitdir(struct repository *repo, const char *path);\n+struct set_gitdir_args {\n+\tconst char *shared_root;\n+\tconst char *object_dir;\n+\tconst char *graft_file;\n+\tconst char *index_file;\n+};\n+\n+extern void repo_set_gitdir(struct repository *repo,\n+\t\t\t const char *root,\n+\t\t\t const struct set_gitdir_args *optional);\n extern void repo_set_worktree(struct repository *repo, const char *path);\n extern void repo_set_hash_algo(struct repository *repo, int algo);\n extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);\ndiff --git a/setup.c b/setup.c\nindex c5d55dcee4..6fac1bb58a 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -1116,8 +1116,7 @@ const char *setup_git_directory_gently(int *nongit_ok)\n \t\t\tconst char *gitdir = getenv(GIT_DIR_ENVIRONMENT);\n \t\t\tif (!gitdir)\n \t\t\t\tgitdir = DEFAULT_GIT_DIR_ENVIRONMENT;\n-\t\t\trepo_set_gitdir(the_repository, gitdir);\n-\t\t\tsetup_git_env();\n+\t\t\tsetup_git_env(gitdir);\n \t\t}\n \t\tif (startup_info->have_repository)\n \t\t\trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340392","messageId":"20180227095846.9238-5-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180227095846.9238-1-pclouds@gmail.com","subject":"[PATCH v2 4/4] repository: delete ignore_env member","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-27T09:58:46Z","receivedAt":"2018-02-27T09:59:42Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This variable was added because the repo_set_gitdir() was created to\ncover both submodule and main repos, but these two are initialized a\nbit differently so ignore_env == 0 means main repo, while ignore_env\n!= 0 is submodules.\n\nSince the difference part (env variables) has been moved out of\nrepo_set_gitdir(), this function works the same way for both repo\ntypes and ignore_env is not needed anymore.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n repository.c | 4 +---\n repository.h | 9 ---------\n 2 files changed, 1 insertion(+), 12 deletions(-)\n\ndiff --git a/repository.c b/repository.c\nindex e326f0fcbc..c555dacad2 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -12,7 +12,7 @@ static struct repository the_repo = {\n \tNULL, NULL, NULL,\n \t&the_index,\n \t&hash_algos[GIT_HASH_SHA1],\n-\t0, 0\n+\t0\n };\n struct repository *the_repository = &the_repo;\n \n@@ -139,8 +139,6 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)\n \tstruct repository_format format;\n \tmemset(repo, 0, sizeof(*repo));\n \n-\trepo->ignore_env = 1;\n-\n \tINIT_LIST_HEAD(&repo->objects.packed_git_mru);\n \n \tif (repo_init_gitdir(repo, gitdir))\ndiff --git a/repository.h b/repository.h\nindex b1da2a6384..07e8971428 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -73,15 +73,6 @@ struct repository {\n \tconst struct git_hash_algo *hash_algo;\n \n \t/* Configurations */\n-\t/*\n-\t * Bit used during initialization to indicate if repository state (like\n-\t * the location of the 'objectdir') should be read from the\n-\t * environment. By default this bit will be set at the begining of\n-\t * 'repo_init()' so that all repositories will ignore the environment.\n-\t * The exception to this is 'the_repository', which doesn't go through\n-\t * the normal 'repo_init()' process.\n-\t */\n-\tunsigned ignore_env:1;\n \n \t/* Indicate if a repository has a different 'commondir' from 'gitdir' */\n \tunsigned different_commondir:1;\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340393","messageId":"20180227095846.9238-3-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180227095846.9238-1-pclouds@gmail.com","subject":"[PATCH v2 2/4] repository.c: delete dead functions","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-27T09:58:44Z","receivedAt":"2018-02-27T09:59:48Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy \n---\n repository.c | 45 ---------------------------------------------\n 1 file changed, 45 deletions(-)\n\ndiff --git a/repository.c b/repository.c\nindex 343efe7282..7654b8ada9 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -16,51 +16,6 @@ static struct repository the_repo = {\n };\n struct repository *the_repository = &the_repo;\n \n-static char *git_path_from_env(const char *envvar, const char *git_dir,\n-\t\t\t const char *path, int fromenv)\n-{\n-\tif (fromenv) {\n-\t\tconst char *value = getenv(envvar);\n-\t\tif (value)\n-\t\t\treturn xstrdup(value);\n-\t}\n-\n-\treturn xstrfmt(\"%s/%s\", git_dir, path);\n-}\n-\n-static int find_common_dir(struct strbuf *sb, const char *gitdir, int fromenv)\n-{\n-\tif (fromenv) {\n-\t\tconst char *value = getenv(GIT_COMMON_DIR_ENVIRONMENT);\n-\t\tif (value) {\n-\t\t\tstrbuf_addstr(sb, value);\n-\t\t\treturn 1;\n-\t\t}\n-\t}\n-\n-\treturn get_common_dir_noenv(sb, gitdir);\n-}\n-\n-static void repo_setup_env(struct repository *repo)\n-{\n-\tstruct strbuf sb = STRBUF_INIT;\n-\n-\trepo->different_commondir = find_common_dir(&sb, repo->gitdir,\n-\t\t\t\t\t\t !repo->ignore_env);\n-\tfree(repo->commondir);\n-\trepo->commondir = strbuf_detach(&sb, NULL);\n-\traw_object_store_clear(&repo->objects);\n-\trepo->objects.objectdir =\n-\t\tgit_path_from_env(DB_ENVIRONMENT, repo->commondir,\n-\t\t\t\t \"objects\", !repo->ignore_env);\n-\tfree(repo->graft_file);\n-\trepo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,\n-\t\t\t\t\t \"info/grafts\", !repo->ignore_env);\n-\tfree(repo->index_file);\n-\trepo->index_file = git_path_from_env(INDEX_ENVIRONMENT, repo->gitdir,\n-\t\t\t\t\t \"index\", !repo->ignore_env);\n-}\n-\n static void expand_base_dir(char **out, const char *in,\n \t\t\t const char *base_dir, const char *def_in)\n {\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340394","messageId":"20180227095846.9238-4-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180227095846.9238-1-pclouds@gmail.com","subject":"[PATCH v2 3/4] sha1_file.c: move delayed getenv(altdb) back to setup_git_env()","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-27T09:58:45Z","receivedAt":"2018-02-27T09:59:53Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"getenv() is supposed to work on the main repository only. This delayed\ngetenv() code in sha1_file.c makes it more difficult to convert\nsha1_file.c to a generic object store that could be used by both\nsubmodule and main repositories.\n\nMove the getenv() back in setup_git_env() where other env vars are\nalso fetched.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n environment.c | 1 +\n object-store.h | 5 ++++-\n object.c | 1 +\n repository.c | 2 ++\n repository.h | 1 +\n sha1_file.c | 6 +-----\n 6 files changed, 10 insertions(+), 6 deletions(-)\n\ndiff --git a/environment.c b/environment.c\nindex 74a2900ddf..47c6e31559 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -174,6 +174,7 @@ void setup_git_env(const char *git_dir)\n \targs.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);\n \targs.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);\n \targs.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);\n+\targs.alternate_db = getenv_safe(&to_free, ALTERNATE_DB_ENVIRONMENT);\n \trepo_set_gitdir(the_repository, git_dir, &args);\n \targv_array_clear(&to_free);\n \ndiff --git a/object-store.h b/object-store.h\nindex afe2f93459..9b1303549b 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -87,6 +87,9 @@ struct raw_object_store {\n \t */\n \tchar *objectdir;\n \n+\t/* Path to extra alternate object database if not NULL */\n+\tchar *alternate_db;\n+\n \tstruct packed_git *packed_git;\n \t/* A most-recently-used ordered version of the packed_git list. */\n \tstruct list_head packed_git_mru;\n@@ -109,7 +112,7 @@ struct raw_object_store {\n \tunsigned packed_git_initialized : 1;\n };\n \n-#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0, 0, 0 }\n+#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0, 0, 0 }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/object.c b/object.c\nindex a7c238339b..5317cfc390 100644\n--- a/object.c\n+++ b/object.c\n@@ -464,6 +464,7 @@ static void free_alt_odbs(struct raw_object_store *o)\n void raw_object_store_clear(struct raw_object_store *o)\n {\n \tFREE_AND_NULL(o->objectdir);\n+\tFREE_AND_NULL(o->alternate_db);\n \n \tfree_alt_odbs(o);\n \to->alt_odb_tail = NULL;\ndiff --git a/repository.c b/repository.c\nindex 7654b8ada9..e326f0fcbc 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -61,6 +61,8 @@ void repo_set_gitdir(struct repository *repo,\n \trepo_set_commondir(repo, o->shared_root);\n \texpand_base_dir(&repo->objects.objectdir, o->object_dir,\n \t\t\trepo->commondir, \"objects\");\n+\tfree(repo->objects.alternate_db);\n+\trepo->objects.alternate_db = xstrdup_or_null(o->alternate_db);\n \texpand_base_dir(&repo->graft_file, o->graft_file,\n \t\t\trepo->commondir, \"info/grafts\");\n \texpand_base_dir(&repo->index_file, o->index_file,\ndiff --git a/repository.h b/repository.h\nindex b5b5d138aa..b1da2a6384 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -94,6 +94,7 @@ struct set_gitdir_args {\n \tconst char *object_dir;\n \tconst char *graft_file;\n \tconst char *index_file;\n+\tconst char *alternate_db;\n };\n \n extern void repo_set_gitdir(struct repository *repo,\ndiff --git a/sha1_file.c b/sha1_file.c\nindex dfc8deec38..ad1cd441e6 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -673,15 +673,11 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \n void prepare_alt_odb(struct repository *r)\n {\n-\tconst char *alt;\n-\n \tif (r->objects.alt_odb_tail)\n \t\treturn;\n \n-\talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n-\n \tr->objects.alt_odb_tail = &r->objects.alt_odb_list;\n-\tlink_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);\n+\tlink_alt_odb_entries(r, r->objects.alternate_db, PATH_SEP, NULL, 0);\n \n \tread_info_alternates(r, r->objects.objectdir, 0);\n }\n-- \n2.16.1.435.g8f24da2e1a\n\n"},{"id":"340395","messageId":"CAPig+cQPz+fr4fCUvLk55xZeEqHr+Y_wST2y9CtyXZohxNV+cA@mail.gmail.com","threadId":"47823","inReplyTo":"20180227095846.9238-1-pclouds@gmail.com","subject":"Re: [PATCH v2 0/4] Delete ignore_env member in struct repository","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-02-27T10:10:08Z","receivedAt":"2018-02-27T10:10:14Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Feb 27, 2018 at 4:58 AM, Nguyễn Thái Ngọc Duy wrote:\n> v2 fixes the incorrect use of consecutive getenv() and adds a comment\n> to clarify the role of old_gitdir\n>\n> diff --git a/environment.c b/environment.c\n> @@ -148,18 +149,34 @@ static char *expand_namespace(const char *raw_namespace)\n> +/* Wrapper of getenv() that returns a strdup value. This value is kept\n> + * in argv to be freed later.\n> + */\n\n /*\n * Comment style.\n */\n\n> +static const char *getenv_safe(struct argv_array *argv, const char *name)\n> +{\n> + const char *value = getenv(name);\n> +\n> + if (!value)\n> + return NULL;\n> +\n> + argv_array_push(argv, value);\n> + return argv->argv[argv->argc - 1];\n> +}\n> +\n> void setup_git_env(const char *git_dir)\n> {\n> const char *shallow_file;\n> const char *replace_ref_base;\n> struct set_gitdir_args args = { NULL };\n> + struct argv_array to_free = ARGV_ARRAY_INIT;\n\nCute. Another example[1] showing that renaming argv_array to something\nelse might be a good idea.\n\n[1]: https://public-inbox.org/git/20180227062128.GG65699@aiede.svl.corp.google.com/\n\n> - args.shared_root = getenv(GIT_COMMON_DIR_ENVIRONMENT);\n> - args.object_dir = getenv(DB_ENVIRONMENT);\n> - args.graft_file = getenv(GRAFT_ENVIRONMENT);\n> - args.index_file = getenv(INDEX_ENVIRONMENT);\n> - args.alternate_db = getenv(ALTERNATE_DB_ENVIRONMENT);\n> + args.shared_root = getenv_safe(&to_free, GIT_COMMON_DIR_ENVIRONMENT);\n> + args.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);\n> + args.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);\n> + args.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);\n> + args.alternate_db = getenv_safe(&to_free, ALTERNATE_DB_ENVIRONMENT);\n> repo_set_gitdir(the_repository, git_dir, &args);\n> + argv_array_clear(&to_free);\n> diff --git a/repository.c b/repository.c\n> @@ -48,6 +48,11 @@ void repo_set_gitdir(struct repository *repo,\n> const char *gitfile = read_gitfile(root);\n> + /*\n> + * repo->gitdir is saved because the caller could pass \"root\"\n> + * that also points to repo->gitdir. We want to keep it alive\n> + * until after xstrdup(root). Then we can free it.\n> + */\n> char *old_gitdir = repo->gitdir;\n\nGood. This comment makes the reasoning clear.\n\n> repo->gitdir = xstrdup(gitfile ? gitfile : root);\n"},{"id":"340415","messageId":"CAGZ79ka5MEk89+1MxR5Fsjg8PSSRQWehxKCNcacA3be9+pQ9+Q@mail.gmail.com","threadId":"47823","inReplyTo":"20180227095846.9238-1-pclouds@gmail.com","subject":"Re: [PATCH v2 0/4] Delete ignore_env member in struct repository","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-27T18:09:38Z","receivedAt":"2018-02-27T18:09:44Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Feb 27, 2018 at 1:58 AM, Nguyễn Thái Ngọc Duy wrote:\n> v2 fixes the incorrect use of consecutive getenv() and adds a comment\n> to clarify the role of old_gitdir\n>\n> Interdiff:\n>\n> diff --git a/environment.c b/environment.c\n> index 95de419de8..47c6e31559 100644\n> --- a/environment.c\n> +++ b/environment.c\n> @@ -14,6 +14,7 @@\n> #include \"fmt-merge-msg.h\"\n> #include \"commit.h\"\n> #include \"object-store.h\"\n> +#include \"argv-array.h\"\n>\n> int trust_executable_bit = 1;\n> int trust_ctime = 1;\n> @@ -148,18 +149,34 @@ static char *expand_namespace(const char *raw_namespace)\n> return strbuf_detach(&buf, NULL);\n> }\n>\n> +/* Wrapper of getenv() that returns a strdup value. This value is kept\n> + * in argv to be freed later.\n> + */\n\n/* comment style, see also Erics reply */\n\nI do not understand why we need to use an argv_array here?\nI see that the push calls xstrdup and then puts it into the array.\nSo to me this looks like a clever way that we can easily free them all\nat once using the predefined argv_array_clear() after calling\nrepo_set_gitdir.\n\nThat makes sense, though is confusing at first; I would have expected\na set_gitdir_args_clear() function that just frees all its strings, whether they\nare NULL or not.\n\nBut given that we are clever with not pushing onto the argv_array in case the\ngetenv returns NULL, this seems to be less work in the usual case of no\nenv variables set.\n\nOk, I think I like it.\n\n> +static const char *getenv_safe(struct argv_array *argv, const char *name)\n\nI would have expected that we already have such a (or similar) helper\nin wrapper.c, but it turns out we always take care of getenv manually,\nand we do not have a wrapper yet.\n\nSo only the comment style remains as a nit.\n\nThanks,\nStefan\n"},{"id":"340434","messageId":"20180227200959.GJ209668@google.com","threadId":"47823","inReplyTo":"20180227095846.9238-2-pclouds@gmail.com","subject":"Re: [PATCH v2 1/4] repository.c: move env-related setup code back to environment.c","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-27T20:09:59Z","receivedAt":"2018-02-27T20:10:19Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/27, Nguyễn Thái Ngọc Duy wrote:\n> It does not make sense that generic repository code contains handling\n> of environment variables, which are specific for the main repository\n> only. Refactor repo_set_gitdir() function to take $GIT_DIR and\n> optionally _all_ other customizable paths. These optional paths can be\n> NULL and will be calculated according to the default directory layout.\n> \n> Note that some dead functions are left behind to reduce diff\n> noise. They will be deleted in the next patch.\n> \n> Signed-off-by: Nguyễn Thái Ngọc Duy \n> ---\n> cache.h | 2 +-\n> environment.c | 29 +++++++++++++++++++++++++---\n> repository.c | 53 +++++++++++++++++++++++++++++++++++++++++++++------\n> repository.h | 11 ++++++++++-\n> setup.c | 3 +--\n> 5 files changed, 85 insertions(+), 13 deletions(-)\n> \n> diff --git a/cache.h b/cache.h\n> index 5717399183..b164a407eb 100644\n> --- a/cache.h\n> +++ b/cache.h\n> @@ -459,7 +459,7 @@ static inline enum object_type object_type(unsigned int mode)\n> */\n> extern const char * const local_repo_env[];\n> \n> -extern void setup_git_env(void);\n> +extern void setup_git_env(const char *git_dir);\n> \n> /*\n> * Returns true iff we have a configured git repository (either via\n> diff --git a/environment.c b/environment.c\n> index ec10b062e6..74a2900ddf 100644\n> --- a/environment.c\n> +++ b/environment.c\n> @@ -14,6 +14,7 @@\n> #include \"fmt-merge-msg.h\"\n> #include \"commit.h\"\n> #include \"object-store.h\"\n> +#include \"argv-array.h\"\n> \n> int trust_executable_bit = 1;\n> int trust_ctime = 1;\n> @@ -148,10 +149,33 @@ static char *expand_namespace(const char *raw_namespace)\n> \treturn strbuf_detach(&buf, NULL);\n> }\n> \n> -void setup_git_env(void)\n> +/* Wrapper of getenv() that returns a strdup value. This value is kept\n> + * in argv to be freed later.\n> + */\n> +static const char *getenv_safe(struct argv_array *argv, const char *name)\n> +{\n> +\tconst char *value = getenv(name);\n> +\n> +\tif (!value)\n> +\t\treturn NULL;\n> +\n> +\targv_array_push(argv, value);\n> +\treturn argv->argv[argv->argc - 1];\n> +}\n> +\n> +void setup_git_env(const char *git_dir)\n> {\n> \tconst char *shallow_file;\n> \tconst char *replace_ref_base;\n> +\tstruct set_gitdir_args args = { NULL };\n> +\tstruct argv_array to_free = ARGV_ARRAY_INIT;\n> +\n> +\targs.shared_root = getenv_safe(&to_free, GIT_COMMON_DIR_ENVIRONMENT);\n> +\targs.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);\n> +\targs.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);\n> +\targs.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);\n> +\trepo_set_gitdir(the_repository, git_dir, &args);\n> +\targv_array_clear(&to_free);\n> \n> \tif (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))\n> \t\tcheck_replace_refs = 0;\n> @@ -301,8 +325,7 @@ int set_git_dir(const char *path)\n> {\n> \tif (setenv(GIT_DIR_ENVIRONMENT, path, 1))\n> \t\treturn error(\"Could not set GIT_DIR to '%s'\", path);\n> -\trepo_set_gitdir(the_repository, path);\n> -\tsetup_git_env();\n> +\tsetup_git_env(path);\n> \treturn 0;\n> }\n> \n> diff --git a/repository.c b/repository.c\n> index a069b1b640..343efe7282 100644\n> --- a/repository.c\n> +++ b/repository.c\n> @@ -61,15 +61,55 @@ static void repo_setup_env(struct repository *repo)\n> \t\t\t\t\t \"index\", !repo->ignore_env);\n> }\n> \n> -void repo_set_gitdir(struct repository *repo, const char *path)\n> +static void expand_base_dir(char **out, const char *in,\n> +\t\t\t const char *base_dir, const char *def_in)\n> {\n> -\tconst char *gitfile = read_gitfile(path);\n> -\tchar *old_gitdir = repo->gitdir;\n> +\tfree(*out);\n> +\tif (in)\n> +\t\t*out = xstrdup(in);\n> +\telse\n> +\t\t*out = xstrfmt(\"%s/%s\", base_dir, def_in);\n> +}\n> +\n> +static void repo_set_commondir(struct repository *repo,\n> +\t\t\t const char *shared_root)\n> +{\n> +\tstruct strbuf sb = STRBUF_INIT;\n> +\n> +\tfree(repo->commondir);\n> \n> -\trepo->gitdir = xstrdup(gitfile ? gitfile : path);\n> -\trepo_setup_env(repo);\n> +\tif (shared_root) {\n> +\t\trepo->different_commondir = 1;\n> +\t\trepo->commondir = xstrdup(shared_root);\n> +\t\treturn;\n> +\t}\n> \n> +\trepo->different_commondir = get_common_dir_noenv(&sb, repo->gitdir);\n> +\trepo->commondir = strbuf_detach(&sb, NULL);\n> +}\n> +\n> +void repo_set_gitdir(struct repository *repo,\n> +\t\t const char *root,\n> +\t\t const struct set_gitdir_args *o)\n> +{\n> +\tconst char *gitfile = read_gitfile(root);\n> +\t/*\n> +\t * repo->gitdir is saved because the caller could pass \"root\"\n> +\t * that also points to repo->gitdir. We want to keep it alive\n> +\t * until after xstrdup(root). Then we can free it.\n> +\t */\n> +\tchar *old_gitdir = repo->gitdir;\n> +\n> +\trepo->gitdir = xstrdup(gitfile ? gitfile : root);\n> \tfree(old_gitdir);\n> +\n> +\trepo_set_commondir(repo, o->shared_root);\n> +\texpand_base_dir(&repo->objects.objectdir, o->object_dir,\n> +\t\t\trepo->commondir, \"objects\");\n> +\texpand_base_dir(&repo->graft_file, o->graft_file,\n> +\t\t\trepo->commondir, \"info/grafts\");\n> +\texpand_base_dir(&repo->index_file, o->index_file,\n> +\t\t\trepo->gitdir, \"index\");\n> }\n> \n> void repo_set_hash_algo(struct repository *repo, int hash_algo)\n> @@ -87,6 +127,7 @@ static int repo_init_gitdir(struct repository *repo, const char *gitdir)\n> \tint error = 0;\n> \tchar *abspath = NULL;\n> \tconst char *resolved_gitdir;\n> +\tstruct set_gitdir_args args = { NULL };\n> \n> \tabspath = real_pathdup(gitdir, 0);\n> \tif (!abspath) {\n> @@ -101,7 +142,7 @@ static int repo_init_gitdir(struct repository *repo, const char *gitdir)\n> \t\tgoto out;\n> \t}\n> \n> -\trepo_set_gitdir(repo, resolved_gitdir);\n> +\trepo_set_gitdir(repo, resolved_gitdir, &args);\n> \n> out:\n> \tfree(abspath);\n> diff --git a/repository.h b/repository.h\n> index fa73ab8e93..b5b5d138aa 100644\n> --- a/repository.h\n> +++ b/repository.h\n> @@ -89,7 +89,16 @@ struct repository {\n> \n> extern struct repository *the_repository;\n> \n> -extern void repo_set_gitdir(struct repository *repo, const char *path);\n> +struct set_gitdir_args {\n> +\tconst char *shared_root;\n\nCan you add a comment explaining what shared_root is? From reading the\ncode it seems to be the common gitdir but its not clear from just\nreading this.\n\n> +\tconst char *object_dir;\n> +\tconst char *graft_file;\n> +\tconst char *index_file;\n> +};\n> +\n> +extern void repo_set_gitdir(struct repository *repo,\n> +\t\t\t const char *root,\n> +\t\t\t const struct set_gitdir_args *optional);\n> extern void repo_set_worktree(struct repository *repo, const char *path);\n> extern void repo_set_hash_algo(struct repository *repo, int algo);\n> extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);\n> diff --git a/setup.c b/setup.c\n> index c5d55dcee4..6fac1bb58a 100644\n> --- a/setup.c\n> +++ b/setup.c\n> @@ -1116,8 +1116,7 @@ const char *setup_git_directory_gently(int *nongit_ok)\n> \t\t\tconst char *gitdir = getenv(GIT_DIR_ENVIRONMENT);\n> \t\t\tif (!gitdir)\n> \t\t\t\tgitdir = DEFAULT_GIT_DIR_ENVIRONMENT;\n> -\t\t\trepo_set_gitdir(the_repository, gitdir);\n> -\t\t\tsetup_git_env();\n> +\t\t\tsetup_git_env(gitdir);\n> \t\t}\n> \t\tif (startup_info->have_repository)\n> \t\t\trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n> -- \n> 2.16.1.435.g8f24da2e1a\n> \n\n-- \nBrandon Williams\n"},{"id":"340435","messageId":"20180227201046.GK209668@google.com","threadId":"47823","inReplyTo":"20180227095846.9238-3-pclouds@gmail.com","subject":"Re: [PATCH v2 2/4] repository.c: delete dead functions","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-27T20:10:46Z","receivedAt":"2018-02-27T20:10:53Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/27, Nguyễn Thái Ngọc Duy wrote:\n> Signed-off-by: Nguyễn Thái Ngọc Duy \n\nThese could technically be removed in the previous patch, it may make\nunderstanding what is happening in the previous patch a bit clearer too.\n\n> ---\n> repository.c | 45 ---------------------------------------------\n> 1 file changed, 45 deletions(-)\n> \n> diff --git a/repository.c b/repository.c\n> index 343efe7282..7654b8ada9 100644\n> --- a/repository.c\n> +++ b/repository.c\n> @@ -16,51 +16,6 @@ static struct repository the_repo = {\n> };\n> struct repository *the_repository = &the_repo;\n> \n> -static char *git_path_from_env(const char *envvar, const char *git_dir,\n> -\t\t\t const char *path, int fromenv)\n> -{\n> -\tif (fromenv) {\n> -\t\tconst char *value = getenv(envvar);\n> -\t\tif (value)\n> -\t\t\treturn xstrdup(value);\n> -\t}\n> -\n> -\treturn xstrfmt(\"%s/%s\", git_dir, path);\n> -}\n> -\n> -static int find_common_dir(struct strbuf *sb, const char *gitdir, int fromenv)\n> -{\n> -\tif (fromenv) {\n> -\t\tconst char *value = getenv(GIT_COMMON_DIR_ENVIRONMENT);\n> -\t\tif (value) {\n> -\t\t\tstrbuf_addstr(sb, value);\n> -\t\t\treturn 1;\n> -\t\t}\n> -\t}\n> -\n> -\treturn get_common_dir_noenv(sb, gitdir);\n> -}\n> -\n> -static void repo_setup_env(struct repository *repo)\n> -{\n> -\tstruct strbuf sb = STRBUF_INIT;\n> -\n> -\trepo->different_commondir = find_common_dir(&sb, repo->gitdir,\n> -\t\t\t\t\t\t !repo->ignore_env);\n> -\tfree(repo->commondir);\n> -\trepo->commondir = strbuf_detach(&sb, NULL);\n> -\traw_object_store_clear(&repo->objects);\n> -\trepo->objects.objectdir =\n> -\t\tgit_path_from_env(DB_ENVIRONMENT, repo->commondir,\n> -\t\t\t\t \"objects\", !repo->ignore_env);\n> -\tfree(repo->graft_file);\n> -\trepo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,\n> -\t\t\t\t\t \"info/grafts\", !repo->ignore_env);\n> -\tfree(repo->index_file);\n> -\trepo->index_file = git_path_from_env(INDEX_ENVIRONMENT, repo->gitdir,\n> -\t\t\t\t\t \"index\", !repo->ignore_env);\n> -}\n> -\n> static void expand_base_dir(char **out, const char *in,\n> \t\t\t const char *base_dir, const char *def_in)\n> {\n> -- \n> 2.16.1.435.g8f24da2e1a\n> \n\n-- \nBrandon Williams\n"},{"id":"340436","messageId":"20180227201250.GL209668@google.com","threadId":"47823","inReplyTo":"20180227095846.9238-4-pclouds@gmail.com","subject":"Re: [PATCH v2 3/4] sha1_file.c: move delayed getenv(altdb) back to setup_git_env()","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-27T20:12:50Z","receivedAt":"2018-02-27T20:12:58Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/27, Nguyễn Thái Ngọc Duy wrote:\n> getenv() is supposed to work on the main repository only. This delayed\n> getenv() code in sha1_file.c makes it more difficult to convert\n> sha1_file.c to a generic object store that could be used by both\n> submodule and main repositories.\n> \n> Move the getenv() back in setup_git_env() where other env vars are\n> also fetched.\n> \n> Signed-off-by: Nguyễn Thái Ngọc Duy \n> ---\n> environment.c | 1 +\n> object-store.h | 5 ++++-\n> object.c | 1 +\n> repository.c | 2 ++\n> repository.h | 1 +\n> sha1_file.c | 6 +-----\n> 6 files changed, 10 insertions(+), 6 deletions(-)\n> \n> diff --git a/environment.c b/environment.c\n> index 74a2900ddf..47c6e31559 100644\n> --- a/environment.c\n> +++ b/environment.c\n> @@ -174,6 +174,7 @@ void setup_git_env(const char *git_dir)\n> \targs.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);\n> \targs.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);\n> \targs.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);\n> +\targs.alternate_db = getenv_safe(&to_free, ALTERNATE_DB_ENVIRONMENT);\n> \trepo_set_gitdir(the_repository, git_dir, &args);\n> \targv_array_clear(&to_free);\n> \n> diff --git a/object-store.h b/object-store.h\n> index afe2f93459..9b1303549b 100644\n> --- a/object-store.h\n> +++ b/object-store.h\n> @@ -87,6 +87,9 @@ struct raw_object_store {\n> \t */\n> \tchar *objectdir;\n> \n> +\t/* Path to extra alternate object database if not NULL */\n> +\tchar *alternate_db;\n> +\n> \tstruct packed_git *packed_git;\n> \t/* A most-recently-used ordered version of the packed_git list. */\n> \tstruct list_head packed_git_mru;\n> @@ -109,7 +112,7 @@ struct raw_object_store {\n> \tunsigned packed_git_initialized : 1;\n> };\n> \n> -#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0, 0, 0 }\n> +#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0, 0, 0 }\n> \n> void raw_object_store_clear(struct raw_object_store *o);\n> \n> diff --git a/object.c b/object.c\n> index a7c238339b..5317cfc390 100644\n> --- a/object.c\n> +++ b/object.c\n> @@ -464,6 +464,7 @@ static void free_alt_odbs(struct raw_object_store *o)\n> void raw_object_store_clear(struct raw_object_store *o)\n> {\n> \tFREE_AND_NULL(o->objectdir);\n> +\tFREE_AND_NULL(o->alternate_db);\n> \n> \tfree_alt_odbs(o);\n> \to->alt_odb_tail = NULL;\n> diff --git a/repository.c b/repository.c\n> index 7654b8ada9..e326f0fcbc 100644\n> --- a/repository.c\n> +++ b/repository.c\n> @@ -61,6 +61,8 @@ void repo_set_gitdir(struct repository *repo,\n> \trepo_set_commondir(repo, o->shared_root);\n> \texpand_base_dir(&repo->objects.objectdir, o->object_dir,\n> \t\t\trepo->commondir, \"objects\");\n> +\tfree(repo->objects.alternate_db);\n> +\trepo->objects.alternate_db = xstrdup_or_null(o->alternate_db);\n\nJust a note that this only works because the object store is embedded in\nthe repository struct, it isn't a pointer to an object store.\n\nPatch looks good.\n\n> \texpand_base_dir(&repo->graft_file, o->graft_file,\n> \t\t\trepo->commondir, \"info/grafts\");\n> \texpand_base_dir(&repo->index_file, o->index_file,\n> diff --git a/repository.h b/repository.h\n> index b5b5d138aa..b1da2a6384 100644\n> --- a/repository.h\n> +++ b/repository.h\n> @@ -94,6 +94,7 @@ struct set_gitdir_args {\n> \tconst char *object_dir;\n> \tconst char *graft_file;\n> \tconst char *index_file;\n> +\tconst char *alternate_db;\n> };\n> \n> extern void repo_set_gitdir(struct repository *repo,\n> diff --git a/sha1_file.c b/sha1_file.c\n> index dfc8deec38..ad1cd441e6 100644\n> --- a/sha1_file.c\n> +++ b/sha1_file.c\n> @@ -673,15 +673,11 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n> \n> void prepare_alt_odb(struct repository *r)\n> {\n> -\tconst char *alt;\n> -\n> \tif (r->objects.alt_odb_tail)\n> \t\treturn;\n> \n> -\talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n> -\n> \tr->objects.alt_odb_tail = &r->objects.alt_odb_list;\n> -\tlink_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);\n> +\tlink_alt_odb_entries(r, r->objects.alternate_db, PATH_SEP, NULL, 0);\n> \n> \tread_info_alternates(r, r->objects.objectdir, 0);\n> }\n> -- \n> 2.16.1.435.g8f24da2e1a\n> \n\n-- \nBrandon Williams\n"},{"id":"340437","messageId":"20180227201445.GM209668@google.com","threadId":"47823","inReplyTo":"20180227095846.9238-5-pclouds@gmail.com","subject":"Re: [PATCH v2 4/4] repository: delete ignore_env member","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-27T20:14:45Z","receivedAt":"2018-02-27T20:14:52Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/27, Nguyễn Thái Ngọc Duy wrote:\n> This variable was added because the repo_set_gitdir() was created to\n> cover both submodule and main repos, but these two are initialized a\n> bit differently so ignore_env == 0 means main repo, while ignore_env\n> != 0 is submodules.\n> \n> Since the difference part (env variables) has been moved out of\n> repo_set_gitdir(), this function works the same way for both repo\n> types and ignore_env is not needed anymore.\n> \n> Signed-off-by: Nguyễn Thái Ngọc Duy \n> ---\n> repository.c | 4 +---\n> repository.h | 9 ---------\n> 2 files changed, 1 insertion(+), 12 deletions(-)\n> \n> diff --git a/repository.c b/repository.c\n> index e326f0fcbc..c555dacad2 100644\n> --- a/repository.c\n> +++ b/repository.c\n> @@ -12,7 +12,7 @@ static struct repository the_repo = {\n> \tNULL, NULL, NULL,\n> \t&the_index,\n> \t&hash_algos[GIT_HASH_SHA1],\n> -\t0, 0\n> +\t0\n> };\n> struct repository *the_repository = &the_repo;\n> \n> @@ -139,8 +139,6 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)\n> \tstruct repository_format format;\n> \tmemset(repo, 0, sizeof(*repo));\n> \n> -\trepo->ignore_env = 1;\n> -\n> \tINIT_LIST_HEAD(&repo->objects.packed_git_mru);\n> \n> \tif (repo_init_gitdir(repo, gitdir))\n> diff --git a/repository.h b/repository.h\n> index b1da2a6384..07e8971428 100644\n> --- a/repository.h\n> +++ b/repository.h\n> @@ -73,15 +73,6 @@ struct repository {\n> \tconst struct git_hash_algo *hash_algo;\n> \n> \t/* Configurations */\n> -\t/*\n> -\t * Bit used during initialization to indicate if repository state (like\n> -\t * the location of the 'objectdir') should be read from the\n> -\t * environment. By default this bit will be set at the begining of\n> -\t * 'repo_init()' so that all repositories will ignore the environment.\n> -\t * The exception to this is 'the_repository', which doesn't go through\n> -\t * the normal 'repo_init()' process.\n> -\t */\n> -\tunsigned ignore_env:1;\n\nNice, glad to see this get a bit less complicated. I'm also very glad I\ndidn't end up using this variable when doing anything else but\ninitializing a repo object otherwise you would have had a bit more\ndifficult time getting rid of it :)\n\nThanks!\n\n> \n> \t/* Indicate if a repository has a different 'commondir' from 'gitdir' */\n> \tunsigned different_commondir:1;\n> -- \n> 2.16.1.435.g8f24da2e1a\n> \n\n-- \nBrandon Williams\n"},{"id":"340502","messageId":"CACsJy8DhZLo2KsA5uYoqJvmJXPz3Gri7Vcpi4huG5pNQW1xLUw@mail.gmail.com","threadId":"47823","inReplyTo":"20180227201250.GL209668@google.com","subject":"Re: [PATCH v2 3/4] sha1_file.c: move delayed getenv(altdb) back to setup_git_env()","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-28T00:58:37Z","receivedAt":"2018-02-28T00:59:13Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Feb 28, 2018 at 3:12 AM, Brandon Williams wrote:\n> On 02/27, Nguyễn Thái Ngọc Duy wrote:\n>> diff --git a/repository.c b/repository.c\n>> index 7654b8ada9..e326f0fcbc 100644\n>> --- a/repository.c\n>> +++ b/repository.c\n>> @@ -61,6 +61,8 @@ void repo_set_gitdir(struct repository *repo,\n>> repo_set_commondir(repo, o->shared_root);\n>> expand_base_dir(&repo->objects.objectdir, o->object_dir,\n>> repo->commondir, \"objects\");\n>> + free(repo->objects.alternate_db);\n>> + repo->objects.alternate_db = xstrdup_or_null(o->alternate_db);\n>\n> Just a note that this only works because the object store is embedded in\n> the repository struct, it isn't a pointer to an object store.\n\nIt is possible to make it work even if object store is not embedded\nthough. From my point of view, this function is about \"give me\n$GIT_DIR and optionally the where all other parts are, if you ignore\ndefault repo layout and tear the repository apart\". We could\ninitialize the object store later when it's created and pass the\nrelevant paths to it. Exactly where it's safe to \"create and\ninitialize object store\" is harder to see now because the whole main\nrepo setup is spread out over many many functions.\n-- \nDuy\n"},{"id":"340521","messageId":"20180228013727.13815-1-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180227095846.9238-1-pclouds@gmail.com","subject":"[PATCH v3 0/4] Delete ignore_env member in struct repository","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-28T01:37:23Z","receivedAt":"2018-02-28T01:37:51Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"v3 fixes comment style. Also since Brandon raised a question about\nshared_root, it's obviously not a good name, so I renamed it to\ncommondir.\n\nI still keep the delete patch 2/4, but I move the repo_setup_env()\ndeletion back to 1/4 so all env logic is in one patch (the\nintroduction of new helper functions in 1/4 and deletion in 2/4 are\nstill diff noise if 2/4 is completely merged back).\n\nInterdiff:\n\ndiff --git a/environment.c b/environment.c\nindex 47c6e31559..b2128c1188 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -149,7 +149,8 @@ static char *expand_namespace(const char *raw_namespace)\n \treturn strbuf_detach(&buf, NULL);\n }\n \n-/* Wrapper of getenv() that returns a strdup value. This value is kept\n+/*\n+ * Wrapper of getenv() that returns a strdup value. This value is kept\n * in argv to be freed later.\n */\n static const char *getenv_safe(struct argv_array *argv, const char *name)\n@@ -170,7 +171,7 @@ void setup_git_env(const char *git_dir)\n \tstruct set_gitdir_args args = { NULL };\n \tstruct argv_array to_free = ARGV_ARRAY_INIT;\n \n-\targs.shared_root = getenv_safe(&to_free, GIT_COMMON_DIR_ENVIRONMENT);\n+\targs.commondir = getenv_safe(&to_free, GIT_COMMON_DIR_ENVIRONMENT);\n \targs.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);\n \targs.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);\n \targs.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);\ndiff --git a/repository.c b/repository.c\nindex c555dacad2..4f44384dde 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -27,15 +27,15 @@ static void expand_base_dir(char **out, const char *in,\n }\n \n static void repo_set_commondir(struct repository *repo,\n-\t\t\t const char *shared_root)\n+\t\t\t const char *commondir)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n \n \tfree(repo->commondir);\n \n-\tif (shared_root) {\n+\tif (commondir) {\n \t\trepo->different_commondir = 1;\n-\t\trepo->commondir = xstrdup(shared_root);\n+\t\trepo->commondir = xstrdup(commondir);\n \t\treturn;\n \t}\n \n@@ -58,7 +58,7 @@ void repo_set_gitdir(struct repository *repo,\n \trepo->gitdir = xstrdup(gitfile ? gitfile : root);\n \tfree(old_gitdir);\n \n-\trepo_set_commondir(repo, o->shared_root);\n+\trepo_set_commondir(repo, o->commondir);\n \texpand_base_dir(&repo->objects.objectdir, o->object_dir,\n \t\t\trepo->commondir, \"objects\");\n \tfree(repo->objects.alternate_db);\ndiff --git a/repository.h b/repository.h\nindex 07e8971428..e05a77a099 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -81,7 +81,7 @@ struct repository {\n extern struct repository *the_repository;\n \n struct set_gitdir_args {\n-\tconst char *shared_root;\n+\tconst char *commondir;\n \tconst char *object_dir;\n \tconst char *graft_file;\n \tconst char *index_file;\n\nNguyễn Thái Ngọc Duy (4):\n repository.c: move env-related setup code back to environment.c\n repository.c: delete dead functions\n sha1_file.c: move delayed getenv(altdb) back to setup_git_env()\n repository: delete ignore_env member\n\n cache.h | 2 +-\n environment.c | 31 +++++++++++++++++--\n object-store.h | 5 ++-\n object.c | 1 +\n repository.c | 84 ++++++++++++++++++++++++--------------------------\n repository.h | 21 +++++++------\n setup.c | 3 +-\n sha1_file.c | 6 +---\n 8 files changed, 87 insertions(+), 66 deletions(-)\n\n-- \n2.16.1.399.g632f88eed1\n\n"},{"id":"340522","messageId":"20180228013727.13815-2-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180228013727.13815-1-pclouds@gmail.com","subject":"[PATCH v3 1/4] repository.c: move env-related setup code back to environment.c","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-28T01:37:24Z","receivedAt":"2018-02-28T01:37:54Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"It does not make sense that generic repository code contains handling\nof environment variables, which are specific for the main repository\nonly. Refactor repo_set_gitdir() function to take $GIT_DIR and\noptionally _all_ other customizable paths. These optional paths can be\nNULL and will be calculated according to the default directory layout.\n\nNote that some dead functions are left behind to reduce diff\nnoise. They will be deleted in the next patch.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n cache.h | 2 +-\n environment.c | 30 +++++++++++++++++++++++---\n repository.c | 59 ++++++++++++++++++++++++++++++++++-----------------\n repository.h | 11 +++++++++-\n setup.c | 3 +--\n 5 files changed, 79 insertions(+), 26 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 5717399183..b164a407eb 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -459,7 +459,7 @@ static inline enum object_type object_type(unsigned int mode)\n */\n extern const char * const local_repo_env[];\n \n-extern void setup_git_env(void);\n+extern void setup_git_env(const char *git_dir);\n \n /*\n * Returns true iff we have a configured git repository (either via\ndiff --git a/environment.c b/environment.c\nindex ec10b062e6..454e435bed 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -14,6 +14,7 @@\n #include \"fmt-merge-msg.h\"\n #include \"commit.h\"\n #include \"object-store.h\"\n+#include \"argv-array.h\"\n \n int trust_executable_bit = 1;\n int trust_ctime = 1;\n@@ -148,10 +149,34 @@ static char *expand_namespace(const char *raw_namespace)\n \treturn strbuf_detach(&buf, NULL);\n }\n \n-void setup_git_env(void)\n+/*\n+ * Wrapper of getenv() that returns a strdup value. This value is kept\n+ * in argv to be freed later.\n+ */\n+static const char *getenv_safe(struct argv_array *argv, const char *name)\n+{\n+\tconst char *value = getenv(name);\n+\n+\tif (!value)\n+\t\treturn NULL;\n+\n+\targv_array_push(argv, value);\n+\treturn argv->argv[argv->argc - 1];\n+}\n+\n+void setup_git_env(const char *git_dir)\n {\n \tconst char *shallow_file;\n \tconst char *replace_ref_base;\n+\tstruct set_gitdir_args args = { NULL };\n+\tstruct argv_array to_free = ARGV_ARRAY_INIT;\n+\n+\targs.commondir = getenv_safe(&to_free, GIT_COMMON_DIR_ENVIRONMENT);\n+\targs.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);\n+\targs.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);\n+\targs.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);\n+\trepo_set_gitdir(the_repository, git_dir, &args);\n+\targv_array_clear(&to_free);\n \n \tif (getenv(NO_REPLACE_OBJECTS_ENVIRONMENT))\n \t\tcheck_replace_refs = 0;\n@@ -301,8 +326,7 @@ int set_git_dir(const char *path)\n {\n \tif (setenv(GIT_DIR_ENVIRONMENT, path, 1))\n \t\treturn error(\"Could not set GIT_DIR to '%s'\", path);\n-\trepo_set_gitdir(the_repository, path);\n-\tsetup_git_env();\n+\tsetup_git_env(path);\n \treturn 0;\n }\n \ndiff --git a/repository.c b/repository.c\nindex a069b1b640..ae117efbf0 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -41,35 +41,55 @@ static int find_common_dir(struct strbuf *sb, const char *gitdir, int fromenv)\n \treturn get_common_dir_noenv(sb, gitdir);\n }\n \n-static void repo_setup_env(struct repository *repo)\n+static void expand_base_dir(char **out, const char *in,\n+\t\t\t const char *base_dir, const char *def_in)\n+{\n+\tfree(*out);\n+\tif (in)\n+\t\t*out = xstrdup(in);\n+\telse\n+\t\t*out = xstrfmt(\"%s/%s\", base_dir, def_in);\n+}\n+\n+static void repo_set_commondir(struct repository *repo,\n+\t\t\t const char *commondir)\n {\n \tstruct strbuf sb = STRBUF_INIT;\n \n-\trepo->different_commondir = find_common_dir(&sb, repo->gitdir,\n-\t\t\t\t\t\t !repo->ignore_env);\n \tfree(repo->commondir);\n+\n+\tif (commondir) {\n+\t\trepo->different_commondir = 1;\n+\t\trepo->commondir = xstrdup(commondir);\n+\t\treturn;\n+\t}\n+\n+\trepo->different_commondir = get_common_dir_noenv(&sb, repo->gitdir);\n \trepo->commondir = strbuf_detach(&sb, NULL);\n-\traw_object_store_clear(&repo->objects);\n-\trepo->objects.objectdir =\n-\t\tgit_path_from_env(DB_ENVIRONMENT, repo->commondir,\n-\t\t\t\t \"objects\", !repo->ignore_env);\n-\tfree(repo->graft_file);\n-\trepo->graft_file = git_path_from_env(GRAFT_ENVIRONMENT, repo->commondir,\n-\t\t\t\t\t \"info/grafts\", !repo->ignore_env);\n-\tfree(repo->index_file);\n-\trepo->index_file = git_path_from_env(INDEX_ENVIRONMENT, repo->gitdir,\n-\t\t\t\t\t \"index\", !repo->ignore_env);\n }\n \n-void repo_set_gitdir(struct repository *repo, const char *path)\n+void repo_set_gitdir(struct repository *repo,\n+\t\t const char *root,\n+\t\t const struct set_gitdir_args *o)\n {\n-\tconst char *gitfile = read_gitfile(path);\n+\tconst char *gitfile = read_gitfile(root);\n+\t/*\n+\t * repo->gitdir is saved because the caller could pass \"root\"\n+\t * that also points to repo->gitdir. We want to keep it alive\n+\t * until after xstrdup(root). Then we can free it.\n+\t */\n \tchar *old_gitdir = repo->gitdir;\n \n-\trepo->gitdir = xstrdup(gitfile ? gitfile : path);\n-\trepo_setup_env(repo);\n-\n+\trepo->gitdir = xstrdup(gitfile ? gitfile : root);\n \tfree(old_gitdir);\n+\n+\trepo_set_commondir(repo, o->commondir);\n+\texpand_base_dir(&repo->objects.objectdir, o->object_dir,\n+\t\t\trepo->commondir, \"objects\");\n+\texpand_base_dir(&repo->graft_file, o->graft_file,\n+\t\t\trepo->commondir, \"info/grafts\");\n+\texpand_base_dir(&repo->index_file, o->index_file,\n+\t\t\trepo->gitdir, \"index\");\n }\n \n void repo_set_hash_algo(struct repository *repo, int hash_algo)\n@@ -87,6 +107,7 @@ static int repo_init_gitdir(struct repository *repo, const char *gitdir)\n \tint error = 0;\n \tchar *abspath = NULL;\n \tconst char *resolved_gitdir;\n+\tstruct set_gitdir_args args = { NULL };\n \n \tabspath = real_pathdup(gitdir, 0);\n \tif (!abspath) {\n@@ -101,7 +122,7 @@ static int repo_init_gitdir(struct repository *repo, const char *gitdir)\n \t\tgoto out;\n \t}\n \n-\trepo_set_gitdir(repo, resolved_gitdir);\n+\trepo_set_gitdir(repo, resolved_gitdir, &args);\n \n out:\n \tfree(abspath);\ndiff --git a/repository.h b/repository.h\nindex fa73ab8e93..f917baa584 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -89,7 +89,16 @@ struct repository {\n \n extern struct repository *the_repository;\n \n-extern void repo_set_gitdir(struct repository *repo, const char *path);\n+struct set_gitdir_args {\n+\tconst char *commondir;\n+\tconst char *object_dir;\n+\tconst char *graft_file;\n+\tconst char *index_file;\n+};\n+\n+extern void repo_set_gitdir(struct repository *repo,\n+\t\t\t const char *root,\n+\t\t\t const struct set_gitdir_args *optional);\n extern void repo_set_worktree(struct repository *repo, const char *path);\n extern void repo_set_hash_algo(struct repository *repo, int algo);\n extern int repo_init(struct repository *repo, const char *gitdir, const char *worktree);\ndiff --git a/setup.c b/setup.c\nindex c5d55dcee4..6fac1bb58a 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -1116,8 +1116,7 @@ const char *setup_git_directory_gently(int *nongit_ok)\n \t\t\tconst char *gitdir = getenv(GIT_DIR_ENVIRONMENT);\n \t\t\tif (!gitdir)\n \t\t\t\tgitdir = DEFAULT_GIT_DIR_ENVIRONMENT;\n-\t\t\trepo_set_gitdir(the_repository, gitdir);\n-\t\t\tsetup_git_env();\n+\t\t\tsetup_git_env(gitdir);\n \t\t}\n \t\tif (startup_info->have_repository)\n \t\t\trepo_set_hash_algo(the_repository, repo_fmt.hash_algo);\n-- \n2.16.1.399.g632f88eed1\n\n"},{"id":"340523","messageId":"20180228013727.13815-3-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180228013727.13815-1-pclouds@gmail.com","subject":"[PATCH v3 2/4] repository.c: delete dead functions","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-28T01:37:25Z","receivedAt":"2018-02-28T01:37:57Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Signed-off-by: Nguyễn Thái Ngọc Duy \n---\n repository.c | 25 -------------------------\n 1 file changed, 25 deletions(-)\n\ndiff --git a/repository.c b/repository.c\nindex ae117efbf0..89e76173a3 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -16,31 +16,6 @@ static struct repository the_repo = {\n };\n struct repository *the_repository = &the_repo;\n \n-static char *git_path_from_env(const char *envvar, const char *git_dir,\n-\t\t\t const char *path, int fromenv)\n-{\n-\tif (fromenv) {\n-\t\tconst char *value = getenv(envvar);\n-\t\tif (value)\n-\t\t\treturn xstrdup(value);\n-\t}\n-\n-\treturn xstrfmt(\"%s/%s\", git_dir, path);\n-}\n-\n-static int find_common_dir(struct strbuf *sb, const char *gitdir, int fromenv)\n-{\n-\tif (fromenv) {\n-\t\tconst char *value = getenv(GIT_COMMON_DIR_ENVIRONMENT);\n-\t\tif (value) {\n-\t\t\tstrbuf_addstr(sb, value);\n-\t\t\treturn 1;\n-\t\t}\n-\t}\n-\n-\treturn get_common_dir_noenv(sb, gitdir);\n-}\n-\n static void expand_base_dir(char **out, const char *in,\n \t\t\t const char *base_dir, const char *def_in)\n {\n-- \n2.16.1.399.g632f88eed1\n\n"},{"id":"340524","messageId":"20180228013727.13815-4-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180228013727.13815-1-pclouds@gmail.com","subject":"[PATCH v3 3/4] sha1_file.c: move delayed getenv(altdb) back to setup_git_env()","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-28T01:37:26Z","receivedAt":"2018-02-28T01:38:00Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"getenv() is supposed to work on the main repository only. This delayed\ngetenv() code in sha1_file.c makes it more difficult to convert\nsha1_file.c to a generic object store that could be used by both\nsubmodule and main repositories.\n\nMove the getenv() back in setup_git_env() where other env vars are\nalso fetched.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n environment.c | 1 +\n object-store.h | 5 ++++-\n object.c | 1 +\n repository.c | 2 ++\n repository.h | 1 +\n sha1_file.c | 6 +-----\n 6 files changed, 10 insertions(+), 6 deletions(-)\n\ndiff --git a/environment.c b/environment.c\nindex 454e435bed..b2128c1188 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -175,6 +175,7 @@ void setup_git_env(const char *git_dir)\n \targs.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);\n \targs.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);\n \targs.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);\n+\targs.alternate_db = getenv_safe(&to_free, ALTERNATE_DB_ENVIRONMENT);\n \trepo_set_gitdir(the_repository, git_dir, &args);\n \targv_array_clear(&to_free);\n \ndiff --git a/object-store.h b/object-store.h\nindex afe2f93459..9b1303549b 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -87,6 +87,9 @@ struct raw_object_store {\n \t */\n \tchar *objectdir;\n \n+\t/* Path to extra alternate object database if not NULL */\n+\tchar *alternate_db;\n+\n \tstruct packed_git *packed_git;\n \t/* A most-recently-used ordered version of the packed_git list. */\n \tstruct list_head packed_git_mru;\n@@ -109,7 +112,7 @@ struct raw_object_store {\n \tunsigned packed_git_initialized : 1;\n };\n \n-#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0, 0, 0 }\n+#define RAW_OBJECT_STORE_INIT(o) { NULL, NULL, NULL, LIST_HEAD_INIT(o.packed_git_mru), NULL, NULL, 0, 0, 0 }\n \n void raw_object_store_clear(struct raw_object_store *o);\n \ndiff --git a/object.c b/object.c\nindex a7c238339b..5317cfc390 100644\n--- a/object.c\n+++ b/object.c\n@@ -464,6 +464,7 @@ static void free_alt_odbs(struct raw_object_store *o)\n void raw_object_store_clear(struct raw_object_store *o)\n {\n \tFREE_AND_NULL(o->objectdir);\n+\tFREE_AND_NULL(o->alternate_db);\n \n \tfree_alt_odbs(o);\n \to->alt_odb_tail = NULL;\ndiff --git a/repository.c b/repository.c\nindex 89e76173a3..b5306ddaa2 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -61,6 +61,8 @@ void repo_set_gitdir(struct repository *repo,\n \trepo_set_commondir(repo, o->commondir);\n \texpand_base_dir(&repo->objects.objectdir, o->object_dir,\n \t\t\trepo->commondir, \"objects\");\n+\tfree(repo->objects.alternate_db);\n+\trepo->objects.alternate_db = xstrdup_or_null(o->alternate_db);\n \texpand_base_dir(&repo->graft_file, o->graft_file,\n \t\t\trepo->commondir, \"info/grafts\");\n \texpand_base_dir(&repo->index_file, o->index_file,\ndiff --git a/repository.h b/repository.h\nindex f917baa584..1b6afd0926 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -94,6 +94,7 @@ struct set_gitdir_args {\n \tconst char *object_dir;\n \tconst char *graft_file;\n \tconst char *index_file;\n+\tconst char *alternate_db;\n };\n \n extern void repo_set_gitdir(struct repository *repo,\ndiff --git a/sha1_file.c b/sha1_file.c\nindex dfc8deec38..ad1cd441e6 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -673,15 +673,11 @@ int foreach_alt_odb(alt_odb_fn fn, void *cb)\n \n void prepare_alt_odb(struct repository *r)\n {\n-\tconst char *alt;\n-\n \tif (r->objects.alt_odb_tail)\n \t\treturn;\n \n-\talt = getenv(ALTERNATE_DB_ENVIRONMENT);\n-\n \tr->objects.alt_odb_tail = &r->objects.alt_odb_list;\n-\tlink_alt_odb_entries(r, alt, PATH_SEP, NULL, 0);\n+\tlink_alt_odb_entries(r, r->objects.alternate_db, PATH_SEP, NULL, 0);\n \n \tread_info_alternates(r, r->objects.objectdir, 0);\n }\n-- \n2.16.1.399.g632f88eed1\n\n"},{"id":"340525","messageId":"20180228013727.13815-5-pclouds@gmail.com","threadId":"47823","inReplyTo":"20180228013727.13815-1-pclouds@gmail.com","subject":"[PATCH v3 4/4] repository: delete ignore_env member","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2018-02-28T01:37:27Z","receivedAt":"2018-02-28T01:38:01Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"This variable was added because the repo_set_gitdir() was created to\ncover both submodule and main repos, but these two are initialized a\nbit differently so ignore_env == 0 means main repo, while ignore_env\n!= 0 is submodules.\n\nSince the difference part (env variables) has been moved out of\nrepo_set_gitdir(), this function works the same way for both repo\ntypes and ignore_env is not needed anymore.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n repository.c | 4 +---\n repository.h | 9 ---------\n 2 files changed, 1 insertion(+), 12 deletions(-)\n\ndiff --git a/repository.c b/repository.c\nindex b5306ddaa2..4f44384dde 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -12,7 +12,7 @@ static struct repository the_repo = {\n \tNULL, NULL, NULL,\n \t&the_index,\n \t&hash_algos[GIT_HASH_SHA1],\n-\t0, 0\n+\t0\n };\n struct repository *the_repository = &the_repo;\n \n@@ -139,8 +139,6 @@ int repo_init(struct repository *repo, const char *gitdir, const char *worktree)\n \tstruct repository_format format;\n \tmemset(repo, 0, sizeof(*repo));\n \n-\trepo->ignore_env = 1;\n-\n \tINIT_LIST_HEAD(&repo->objects.packed_git_mru);\n \n \tif (repo_init_gitdir(repo, gitdir))\ndiff --git a/repository.h b/repository.h\nindex 1b6afd0926..e05a77a099 100644\n--- a/repository.h\n+++ b/repository.h\n@@ -73,15 +73,6 @@ struct repository {\n \tconst struct git_hash_algo *hash_algo;\n \n \t/* Configurations */\n-\t/*\n-\t * Bit used during initialization to indicate if repository state (like\n-\t * the location of the 'objectdir') should be read from the\n-\t * environment. By default this bit will be set at the begining of\n-\t * 'repo_init()' so that all repositories will ignore the environment.\n-\t * The exception to this is 'the_repository', which doesn't go through\n-\t * the normal 'repo_init()' process.\n-\t */\n-\tunsigned ignore_env:1;\n \n \t/* Indicate if a repository has a different 'commondir' from 'gitdir' */\n \tunsigned different_commondir:1;\n-- \n2.16.1.399.g632f88eed1\n\n"},{"id":"340587","messageId":"20180228181201.GA87122@google.com","threadId":"47823","inReplyTo":"20180228013727.13815-1-pclouds@gmail.com","subject":"Re: [PATCH v3 0/4] Delete ignore_env member in struct repository","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2018-02-28T18:12:01Z","receivedAt":"2018-02-28T18:12:10Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 02/28, Nguyễn Thái Ngọc Duy wrote:\n> v3 fixes comment style. Also since Brandon raised a question about\n> shared_root, it's obviously not a good name, so I renamed it to\n> commondir.\n\nThanks, its a bit clearer and more consistent with the rest of the\nterminology we already have :)\n\n> \n> I still keep the delete patch 2/4, but I move the repo_setup_env()\n> deletion back to 1/4 so all env logic is in one patch (the\n> introduction of new helper functions in 1/4 and deletion in 2/4 are\n> still diff noise if 2/4 is completely merged back).\n> \n> Interdiff:\n> \n> diff --git a/environment.c b/environment.c\n> index 47c6e31559..b2128c1188 100644\n> --- a/environment.c\n> +++ b/environment.c\n> @@ -149,7 +149,8 @@ static char *expand_namespace(const char *raw_namespace)\n> \treturn strbuf_detach(&buf, NULL);\n> }\n> \n> -/* Wrapper of getenv() that returns a strdup value. This value is kept\n> +/*\n> + * Wrapper of getenv() that returns a strdup value. This value is kept\n> * in argv to be freed later.\n> */\n> static const char *getenv_safe(struct argv_array *argv, const char *name)\n> @@ -170,7 +171,7 @@ void setup_git_env(const char *git_dir)\n> \tstruct set_gitdir_args args = { NULL };\n> \tstruct argv_array to_free = ARGV_ARRAY_INIT;\n> \n> -\targs.shared_root = getenv_safe(&to_free, GIT_COMMON_DIR_ENVIRONMENT);\n> +\targs.commondir = getenv_safe(&to_free, GIT_COMMON_DIR_ENVIRONMENT);\n> \targs.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);\n> \targs.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);\n> \targs.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);\n> diff --git a/repository.c b/repository.c\n> index c555dacad2..4f44384dde 100644\n> --- a/repository.c\n> +++ b/repository.c\n> @@ -27,15 +27,15 @@ static void expand_base_dir(char **out, const char *in,\n> }\n> \n> static void repo_set_commondir(struct repository *repo,\n> -\t\t\t const char *shared_root)\n> +\t\t\t const char *commondir)\n> {\n> \tstruct strbuf sb = STRBUF_INIT;\n> \n> \tfree(repo->commondir);\n> \n> -\tif (shared_root) {\n> +\tif (commondir) {\n> \t\trepo->different_commondir = 1;\n> -\t\trepo->commondir = xstrdup(shared_root);\n> +\t\trepo->commondir = xstrdup(commondir);\n> \t\treturn;\n> \t}\n> \n> @@ -58,7 +58,7 @@ void repo_set_gitdir(struct repository *repo,\n> \trepo->gitdir = xstrdup(gitfile ? gitfile : root);\n> \tfree(old_gitdir);\n> \n> -\trepo_set_commondir(repo, o->shared_root);\n> +\trepo_set_commondir(repo, o->commondir);\n> \texpand_base_dir(&repo->objects.objectdir, o->object_dir,\n> \t\t\trepo->commondir, \"objects\");\n> \tfree(repo->objects.alternate_db);\n> diff --git a/repository.h b/repository.h\n> index 07e8971428..e05a77a099 100644\n> --- a/repository.h\n> +++ b/repository.h\n> @@ -81,7 +81,7 @@ struct repository {\n> extern struct repository *the_repository;\n> \n> struct set_gitdir_args {\n> -\tconst char *shared_root;\n> +\tconst char *commondir;\n> \tconst char *object_dir;\n> \tconst char *graft_file;\n> \tconst char *index_file;\n> \n> Nguyễn Thái Ngọc Duy (4):\n> repository.c: move env-related setup code back to environment.c\n> repository.c: delete dead functions\n> sha1_file.c: move delayed getenv(altdb) back to setup_git_env()\n> repository: delete ignore_env member\n> \n> cache.h | 2 +-\n> environment.c | 31 +++++++++++++++++--\n> object-store.h | 5 ++-\n> object.c | 1 +\n> repository.c | 84 ++++++++++++++++++++++++--------------------------\n> repository.h | 21 +++++++------\n> setup.c | 3 +-\n> sha1_file.c | 6 +---\n> 8 files changed, 87 insertions(+), 66 deletions(-)\n> \n> -- \n> 2.16.1.399.g632f88eed1\n> \n\n-- \nBrandon Williams\n"},{"id":"340595","messageId":"xmqqefl5kkn3.fsf@gitster-ct.c.googlers.com","threadId":"47823","inReplyTo":"20180228013727.13815-1-pclouds@gmail.com","subject":"Re: [PATCH v3 0/4] Delete ignore_env member in struct repository","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-02-28T18:57:04Z","receivedAt":"2018-02-28T18:57:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyễn Thái Ngọc Duy writes:\n\n> v3 fixes comment style. Also since Brandon raised a question about\n> shared_root, it's obviously not a good name, so I renamed it to\n> commondir.\n>\n> I still keep the delete patch 2/4, but I move the repo_setup_env()\n> deletion back to 1/4 so all env logic is in one patch (the\n> introduction of new helper functions in 1/4 and deletion in 2/4 are\n> still diff noise if 2/4 is completely merged back).\n\nThanks. After replacing sb/object-store with the last one posted\nand re-tweaking resolution of conflicts with ds/commit-graph that is\nin flight, I think I can squeeze this topic in. Among the three, I\nfind that this kind of clean-up is the most interesting/urgent thing\nwe need before we can move forward ;-)\n\n\n\n>\n> Interdiff:\n>\n> diff --git a/environment.c b/environment.c\n> index 47c6e31559..b2128c1188 100644\n> --- a/environment.c\n> +++ b/environment.c\n> @@ -149,7 +149,8 @@ static char *expand_namespace(const char *raw_namespace)\n> \treturn strbuf_detach(&buf, NULL);\n> }\n> \n> -/* Wrapper of getenv() that returns a strdup value. This value is kept\n> +/*\n> + * Wrapper of getenv() that returns a strdup value. This value is kept\n> * in argv to be freed later.\n> */\n> static const char *getenv_safe(struct argv_array *argv, const char *name)\n> @@ -170,7 +171,7 @@ void setup_git_env(const char *git_dir)\n> \tstruct set_gitdir_args args = { NULL };\n> \tstruct argv_array to_free = ARGV_ARRAY_INIT;\n> \n> -\targs.shared_root = getenv_safe(&to_free, GIT_COMMON_DIR_ENVIRONMENT);\n> +\targs.commondir = getenv_safe(&to_free, GIT_COMMON_DIR_ENVIRONMENT);\n> \targs.object_dir = getenv_safe(&to_free, DB_ENVIRONMENT);\n> \targs.graft_file = getenv_safe(&to_free, GRAFT_ENVIRONMENT);\n> \targs.index_file = getenv_safe(&to_free, INDEX_ENVIRONMENT);\n> diff --git a/repository.c b/repository.c\n> index c555dacad2..4f44384dde 100644\n> --- a/repository.c\n> +++ b/repository.c\n> @@ -27,15 +27,15 @@ static void expand_base_dir(char **out, const char *in,\n> }\n> \n> static void repo_set_commondir(struct repository *repo,\n> -\t\t\t const char *shared_root)\n> +\t\t\t const char *commondir)\n> {\n> \tstruct strbuf sb = STRBUF_INIT;\n> \n> \tfree(repo->commondir);\n> \n> -\tif (shared_root) {\n> +\tif (commondir) {\n> \t\trepo->different_commondir = 1;\n> -\t\trepo->commondir = xstrdup(shared_root);\n> +\t\trepo->commondir = xstrdup(commondir);\n> \t\treturn;\n> \t}\n> \n> @@ -58,7 +58,7 @@ void repo_set_gitdir(struct repository *repo,\n> \trepo->gitdir = xstrdup(gitfile ? gitfile : root);\n> \tfree(old_gitdir);\n> \n> -\trepo_set_commondir(repo, o->shared_root);\n> +\trepo_set_commondir(repo, o->commondir);\n> \texpand_base_dir(&repo->objects.objectdir, o->object_dir,\n> \t\t\trepo->commondir, \"objects\");\n> \tfree(repo->objects.alternate_db);\n> diff --git a/repository.h b/repository.h\n> index 07e8971428..e05a77a099 100644\n> --- a/repository.h\n> +++ b/repository.h\n> @@ -81,7 +81,7 @@ struct repository {\n> extern struct repository *the_repository;\n> \n> struct set_gitdir_args {\n> -\tconst char *shared_root;\n> +\tconst char *commondir;\n> \tconst char *object_dir;\n> \tconst char *graft_file;\n> \tconst char *index_file;\n>\n> Nguyễn Thái Ngọc Duy (4):\n> repository.c: move env-related setup code back to environment.c\n> repository.c: delete dead functions\n> sha1_file.c: move delayed getenv(altdb) back to setup_git_env()\n> repository: delete ignore_env member\n>\n> cache.h | 2 +-\n> environment.c | 31 +++++++++++++++++--\n> object-store.h | 5 ++-\n> object.c | 1 +\n> repository.c | 84 ++++++++++++++++++++++++--------------------------\n> repository.h | 21 +++++++------\n> setup.c | 3 +-\n> sha1_file.c | 6 +---\n> 8 files changed, 87 insertions(+), 66 deletions(-)\n"},{"id":"340846","messageId":"CACsJy8Dpzj2XoaC6WjahKPiOhpY9i9iAPwPjmDmw1aFwbEw2OA@mail.gmail.com","threadId":"47823","inReplyTo":"xmqqfu5nqrf8.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 2/2] Revert \"repository: pre-initialize hash algo pointer\"","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-03-03T01:51:34Z","receivedAt":"2018-03-03T01:52:10Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Feb 27, 2018 at 6:09 AM, Junio C Hamano wrote:\n> Duy Nguyen writes:\n>\n>> Yes that's the intention. But after writing cover letter for v2 and\n>> sending it out, it looks to me that this thing must stay until all our\n>> code is converted to using the_hash_algo (I don't know if there are\n>> more to convert or it's finished already). So an alternative is we do\n>> the opposite: default to GIT_HASH_SHA1, but when an env variable is\n>> set, reset it back to NULL. This env variable will _always_ be set by\n>> the test suite to help us catch problems.\n>\n> If we were to do the \"do not blindly initialize and always\n> initialize explicitly for debugging\" (which I doubt is a good\n> direction to go in), then what you outlined above certainly is a\n> less evil way to do so than what the patch under discussion did, I\n> would think.\n\nSo v3 [1] should be in good condition to go to 'pu', I think?\n\n[1] https://public-inbox.org/git/20180225111840.16421-1-pclouds@gmail.com/\n-- \nDuy\n"}]}