[PATCH 0/3] WIP list-objects and pack-objects for partial clone

From: Jeff Hostetler <jeffhost@microsoft.com>

This WIP is a follow up to earlier patches to teach pack-objects to omit large blobs from packfiles. This doesn't attempt to solve the whole end-to-end problem of partial/sparse clone/fetch or that of the client operating with missing blobs. This WIP is for now limited to building the packfile with omitted blobs and hopefully can mesh nicely with Jonathan Tan's work in [3].

It supports filtering by size while always including blobs associated with ".git*" paths, something we both proposed in [1] and [3].

The approach here differs from [1] and [3] in that it extends traverse_commit_list() to allow custom blob filtering using a new callback provided by pack-objects. This should make it easier to do other filters laters. Part of this based upon Peff's suggestion about rev-list in [2]. I have not updated the rev-list command, but rather the routines in list-objects.c that it calls. Jonathan's ideas in [3] to build and send the omitted blobs list means that I think we need pack-objects.c manage the filter-proc used here.

I considered, but omitted from this version, ideas to allow the filter-proc to know of the process_tree() boundaries which might let pack-objects filter by sub-tree (think sparse-checkout) as suggested in [4] and various replies.

[1] https://public-inbox.org/git/1488994685-37403-3-git-send-email-jeffhost@microsoft.com/ [2] https://public-inbox.org/git/20170309073117.g3br5btsfwntcdpe@sigill.intra.peff.net/ [3] https://public-inbox.org/git/cover.1496361873.git.jonathantanmy@google.com/ [4] https://public-inbox.org/git/20170602232508.GA21733@aiede.mtv.corp.google.com/

Jeff Hostetler (3):
  list-objects: add filter_blob to traverse_commit_list
  pack-objects: WIP add max-blob-size filtering
  pack-objects: add t5317 to test max-blob-size

 builtin/pack-objects.c                 | 76 +++++++++++++++++++++++++++++++++-
 list-objects.c                         | 39 +++++++++++++++--
 list-objects.h                         |  8 ++++
 t/t5317-pack-objects-blob-filtering.sh | 68 ++++++++++++++++++++++++++++++
 4 files changed, 186 insertions(+), 5 deletions(-)
 create mode 100644 t/t5317-pack-objects-blob-filtering.sh