git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Borrowing objects from nearby repositories

From
Phil Hord <phil.hord@gmail.com>
Date
Mar 23, 2014, 18:04 UTC
Message-ID
<CABURp0rKz9s7aPx_6ucTQ5C8NpPZMcJL3jaB_v_rQaBE+sFt1Q@mail.gmail.com>
In-Reply-To
<BFF5FBC7-8F53-4958-8D56-90EADD3AD626@kellerfarm.com>
On Tue, Mar 11, 2014 at 11:37 PM, Andrew Keller <andrew@kellerfarm.com> wrote:
Show 7 quoted lines
> I am considering developing a new feature, and I'd like to poll the group for opinions.
>
> Background: A couple years ago, I wrote a set of scripts that speed up cloning of frequently used repositories.  The scripts utilize a bare Git repository located at a known location, and automate providing a --reference parameter to `git clone` and `git submodule update`.  Recently, some coworkers of mine expressed an interest in using the scripts, so I published the current version of my scripts, called `git repocache`, described at the bottom of <https://github.com/andrewkeller/ak-git-tools>.
>
> Slowly, it has occurred to me that this feature, or something similar to it, may be worth adding to Git, so I've been thinking about the best approach.  Here's my best idea so far:
>
> 1)  Introduce '--borrow' to `git-fetch`.  This would behave similarly to '--reference', except that it operates on a temporary basis, and does not assume that the reference repository will exist after the operation completes, so any used objects are copied into the local objects database.  In theory, this mechanism would be distinct from '--reference', so if both are used, some objects would be copied, and some objects would be accessible via a reference repository referenced by the alternates file.

Interesting. I do something similar on my CI Server to reduce workload on Gerrit. Having a built-in to support submodules would be nice. Currently my script does this:

MIRROR=/path/to/local/mirror NEW=ssh://gerrit-server git clone ${MIRROR}/project && cd project

#-- Init/update submodules from our local mirror if possible git submodule update --recursive --init

#-- Switch to the remote server URL git config remote.origin.url $(git config remote.origin.url|sed -e "s|^${MIRROR}|${NEW}|") git submodule sync #--recursive ; recursive not supported :-[

#-- Checkout remote updates git pull --ff-only --recurse-submodules origin ${BRANCH} git submodule update --recursive --init

Is that about the same as you are aiming for?
> 2)  Teach `git fetch` to read 'repocache.path' (or a better-named configuration), and use it to automatically activate borrowing.

Seems like this could be trouble if a local repo is coincidentally named the same as some unrelated repo you want to clone. But I can see the value.

What about something similar to url.insteadOf? Maybe 'url.${SERVER}.autoBorrow = ${MIRROR}', with replacement semantics similar to insteadOf.

> 3)  For consistency, `git clone`, `git pull`, and `git submodule update` should probably all learn '--borrow', and forward it to `git fetch`.
>
> 4)  In some scenarios, it may be necessary to temporarily not automatically borrow, so `git fetch`, and everything that calls it may need an argument to do that.
--no-borrow
Phil
Previous: Andrew KellerNext: Ævar Arnfjörð Bjarmason
Message 2 of 10 in “Borrowing objects from nearby repositories”
  1. Andrew KellerMar 12, 2014
  2. Phil HordMar 23, 2014
  3. Ævar Arnfjörð BjarmasonMar 24, 2014
  4. Andrew KellerMar 25, 2014
  5. Junio C HamanoMar 25, 2014
  6. Junio C HamanoMar 25, 2014
  7. Andrew KellerMar 26, 2014
  8. Junio C HamanoMar 26, 2014
  9. Andrew KellerMar 28, 2014
  10. Junio C HamanoMar 28, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.