The design of refs backends, linked worktrees and submodules

I've started working on fixing the "git gc" issue with multiple worktrees, which brings me back to this. Just some thoughts. Comments are really appreciated.

In the current code, files backend has special cases for both submodules (explicitly) and linked worktrees (hidden behind git_path). But if a backend has to handle this stuff, all future backends have to too. Which does not sound great. Imagine we have "something" in addition to worktrees and submodules in future, then all backends have to learn about it.

So how about we move worktree and submodule support back to front-end?

The backend deals with refs, period. The file-based backend will be given a directory where refs live in and it work on that. Backends do not use git_path(). Backends do not care about $GIT_DIR. Access to odb (e.g. sha-1 validation) if needed is abstracted out via a set of callbacks. This allows submodules to give access to submodule's separate odb. And it's getting close to the "struct repository" mentioned somewhere in refs "TODO" comments, even though we are nowhere close to introducing that struct.

How does that sound? In particular, is there anything wrong, or unrealistic, with that?

For that to work, I'll probably need to add a "composite" ref_store that combines two file-based backends (for per-repo and per-worktree refs) to represent a unified ref store. I think your work on ref iterator makes way for that. A bit worried about transactions though, because I think per-repo and per-worktree updates will be separated in two transactions. But that's probably ok because future backends, like lmdb, will have to go through the same route anyway.