git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC] Extending git-replace

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Jan 14, 2020, 19:11 UTC
Message-ID
<20200114191156.122421-1-jonathantanmy@google.com>
In-Reply-To
<CABPp-BGy3qu_Rd4epore0wLyoh1fg0UH5EAV27shKJ=kLWX4FA@mail.gmail.com>
Show 17 quoted lines
> 2) Some folks might be okay with a clone that won't pass fsck or
> prune, at least in special circumstances.  We're actually doing that
> on purpose to deal with one of our large repositories.  We don't
> provide that to normal developers, but we do use "cheap, fake clones"
> in our CI systems.  These slim clones have 99% of all objects, but
> happen to be missing the really big ones, resulting in only needing
> 1/7 of the time to download.  (And no, don't try to point out shallow
> clones to me.  I hate those things, they're an awful hack, *and* they
> don't work for us.  It's nice getting all commit history, all trees,
> and most blobs including all for at least the last two years while
> still saving lots of space.)
> 
> [For the curious, I did make a simple script to create these "cheap,
> fake clones" for repositories of interest.  See
> https://github.com/newren/sequester-old-big-blobs.  But they are
> definitely a hack with some sharp corners, with failing fsck and
> prunes only being part of the story.]

If you want to reduce the sharpness of the corners, it might be possible to designate the pack with missing blobs as a promisor pack (add a .promisor file - which is just like the .keep file except s/keep/promisor/) and a fake promisor remote. That will make fsck and repack (GC) work.

Show 11 quoted lines
> 3) Back to your idea...
> 
> What you're proposing actually sounds very similar to partial clones,
> whose idea is to make it okay to download a subset of history.  The
> primary problems with partial clones are (a) they are still under
> development and are just experimental, (b) they are currently
> implemented with a "promisor" mode, meaning that if a command tries to
> run over any piece of missing data then the command pauses while the
> objects are downloaded from the server.  I want an offline mode (even
> if I'm online) where only explicit downloading from the server (clone,
> fetch, etc.) occurs.

David Turner had an idea of what could be done (instead of fetching) in such an offline mode [1], so I replied there.

[1] https://lore.kernel.org/git/d4361b6d34513a3fdefa564d10269f60d4732208.camel@novalis.org/
Show 8 quoted lines
> Instead of inventing yet another partial-clone-like thing, it'd be
> nice if your new mechanism could just be implemented in terms of
> partial clones, extending them as you need.  I don't like the idea of
> supporting multiple competing implementations of partial clones
> withing git.git, but if it's just some extensions of the existing
> capability then it sounds great.  But you may want to talk with
> Jonathan Tan if you want to go this route (cc'd), since he's the
> partial clone expert.
Ah, thanks for your kind words.
Previous: Elijah NewrenNext: Kaushik Srenevasan
Message 3 of 9 in “[RFC] Extending git-replace”
  1. Kaushik SrenevasanJan 14, 2020
  2. Elijah NewrenJan 14, 2020
  3. Jonathan TanJan 14, 2020
  4. Kaushik SrenevasanJan 16, 2020
  5. David TurnerJan 14, 2020
  6. Jonathan TanJan 14, 2020
  7. Elijah NewrenJan 14, 2020
  8. Jonathan TanJan 14, 2020
  9. Elijah NewrenJan 14, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.