[PATCH 00/28] Create a reference backend for packed refs

This patch series continues the saga of picking apart the code for handling packed references from the code for handling loose references, all in preparation for making big changes to how the packed-ref reading and writing works as described in [1]. As a reminder, the final goal is to read the "packed-refs" file using mmap, parsing it on the fly instead of storing it into an in-memory `ref_cache`, and to read and parse only the parts of the file that are actually needed, giving a big speedup for many operations in repositories that have lots of refs.

In this episode, we create a `packed_ref_store` class, implementing part of the `ref_store` API, that represents the packed references within a repository. The `files_ref_store` now contains an instance of `packed_ref_store` and delegates to it for the operations that have to touch the packed refs.

After this patch series, `packed_ref_store` supports:

* Iteration
* `peel_ref`
* `pack_refs` (they're already packed, so it's a NOOP)
* `read_raw_ref`

A future patch series will add support for:

* Reference transactions (`transaction_prepare`, `transaction_finish`,
  `transaction_abort`, `initial_transaction_commit`)
* `delete_refs`

Operations that `packed_ref_store` will probably never support:

* `create_symref`
* `rename_ref` (could be supported, but is probably not useful)
* Reflog-related operations

In addition, all of the packed-refs related code has been moved to a new module, "refs/packed-backend.{c,h}". This includes some functions that are still called by `files_ref_store` directly to update the packed refs.

The patch series is long, but I think relatively straightforward. In particular, patches 2-14 are quite mechanical. Its main point is to separate concerns, but it does bring one end-user advantage: if there is a problem parsing the "packed-refs" file, we now report an error and die. The old code just ignored lines that it didn't understand.

I've developed these patches on top of master plus the following patches, which are followups to mh/packed-refs-store-prep:

* lock_packed_refs(): fix cache validity check
* for_each_bisect_ref(): don't trim refnames

The patches can also be obtained from my GitHub fork [2] as branch "packed-ref-store".

Michael

[1] http://public-inbox.org/git/cover.1490026594.git.mhagger@alum.mit.edu/ [2] https://github.com/mhagger/git

Michael Haggerty (28):
  add_packed_ref(): teach function to overwrite existing refs
  packed_ref_store: new struct
  packed_ref_store: move `packed_refs_path` here
  packed_ref_store: move `packed_refs_lock` member here
  clear_packed_ref_cache(): take a `packed_ref_store *` parameter
  validate_packed_ref_cache(): take a `packed_ref_store *` parameter
  get_packed_ref_cache(): take a `packed_ref_store *` parameter
  get_packed_refs(): take a `packed_ref_store *` parameter
  add_packed_ref(): take a `packed_ref_store *` parameter
  lock_packed_refs(): take a `packed_ref_store *` parameter
  commit_packed_refs(): take a `packed_ref_store *` parameter
  rollback_packed_refs(): take a `packed_ref_store *` parameter
  get_packed_ref(): take a `packed_ref_store *` parameter
  repack_without_refs(): take a `packed_ref_store *` parameter
  packed_peel_ref(): new function, extracted from `files_peel_ref()`
  packed_ref_store: support iteration
  packed_read_raw_ref(): new function, replacing `resolve_packed_ref()`
  packed-backend: new module for handling packed references
  packed_ref_store: make class into a subclass of `ref_store`
  commit_packed_refs(): report errors rather than dying
  commit_packed_refs(): use a staging file separate from the lockfile
  packed_refs_lock(): function renamed from lock_packed_refs()
  packed_refs_lock(): report errors via a `struct strbuf *err`
  packed_refs_unlock(), packed_refs_is_locked(): new functions
  clear_packed_ref_cache(): don't protest if the lock is held
  commit_packed_refs(): remove call to `packed_refs_unlock()`
  repack_without_refs(): don't lock or unlock the packed refs
  read_packed_refs(): die if `packed-refs` contains bogus data

 Makefile              |   1 +
 refs.c                |  18 ++
 refs/files-backend.c  | 619 ++++-------------------------------
 refs/packed-backend.c | 868 ++++++++++++++++++++++++++++++++++++++++++++++++++
 refs/packed-backend.h |  25 ++
 refs/refs-internal.h  |  10 +
 6 files changed, 981 insertions(+), 560 deletions(-)
 create mode 100644 refs/packed-backend.c
 create mode 100644 refs/packed-backend.h