Skipping history and save bandwidth: how can I jump between shallow clones, without sending unused blobs

Robin H. Johnson· Dec 18, 2018, 21:56 UTC · lore

I think this is encapsulated in the v2/promisor work, but wanted to check how close that was to fruition, and that it would indeed be possible.

This would enable replacement of any workflow that presently uses rsync to update.

If I have a clone (ideally shallow already) at A, I'd like to update it to the latest remote tip at F, ALSO at depth 1, without fetching the intermediate history (B..E), or blobs unique to the intermediate history [and no longer referenced from any tree at F].

I critically want to ensure:
- that the remote does NOT send any blobs that I already have.
- that the remote does NOT send any blobs that are not reachable from
  the new tip (e.g. blobs that existed between the old state and the new
  state, but aren't needed anymore).

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

Duy Nguyen· Dec 19, 2018, 11:30 UTC · re: Robin H. Johnson · lore

Re: Skipping history and save bandwidth: how can I jump between shallow clones, without sending unused blobs

On Wed, Dec 19, 2018 at 12:15 AM Robin H. Johnson <robbat2@gentoo.org> wrote:

Show 18 quoted lines

>
> I think this is encapsulated in the v2/promisor work, but wanted to
> check how close that was to fruition, and that it would indeed be
> possible.
>
> This would enable replacement of any workflow that presently uses rsync
> to update.
>
> If I have a clone (ideally shallow already) at A, I'd like to update it
> to the latest remote tip at F, ALSO at depth 1, without fetching the
> intermediate history (B..E), or blobs unique to the intermediate history
> [and no longer referenced from any tree at F].
>
> I critically want to ensure:
> - that the remote does NOT send any blobs that I already have.
> - that the remote does NOT send any blobs that are not reachable from
>   the new tip (e.g. blobs that existed between the old state and the new
>   state, but aren't needed anymore).

This is what we do now (I've created a small repo to test it again). Are you seeing that we transfer more than needed?

Show 7 quoted lines

>
> --
> Robin Hugh Johnson
> Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
> E-Mail   : robbat2@gentoo.org
> GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
> GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

-- 
Duy