# Re: is hosting a read-mostly git repo on a distributed file system practical?

2 messages from 2011-04-13 to 2011-04-13. Participants: George Spelvin, Jon Seymour.
Thread: https://gitlist.dev/t/27079

## George Spelvin, 2011-04-13 03:47

Subject: Re: is hosting a read-mostly git repo on a distributed file system practical?
Message-ID: <20110413034715.15553.qmail@science.horizon.com>
URL: https://gitlist.dev/e/20110413034715.15553.qmail%40science.horizon.com

```
> All clients, including the client that occasionally updates the
> read-mostly repo would be mounting the DFS as a local file system. My
> environment is one where DFS is easy, but establishing a shared server
> is more complicated (ie. bureaucratic).

> I guess I am prepared to put up with a slow initial clone (my developer
> pool will be relatively stable and pulling from a peer via git: or ssh:
> will usually be acceptable for this occasional need).

> What I am most interested in is the incremental performance. Can my
> integrator, who occasionally updates the shared repo, avoid automatically
> repacking it (and hence taking the whole of repo latency hit) and can
> my developers who are pulling the updates do so reliably without a whole
> of repo scan?

I think the answers are yes, but I have to make a vouple of things clear:
* You can *definitely* control repack behaviour.  .keep files are the
  simplest way to prevent repacking.
* Are you talking about hosting only a "bare" repository, or one with
  the unpacked source tree as well?  If you try to run git commands on
  a large network-mounted source tree, things can get more than a bit
  sluggish; git recursively stats the whole tree fairly frequently.
  (There are ways to precent that, notably core.ignoreStat, but they
  make it less friendly.)
* You can clone from a repository mounted on the file system just as
  easily as you can from a network server.  So there's no need to set
  up a server if you find it onconvenient.
* Normally, the developers will clone from the integrator's repository
  before doing anything, so the source tree, and any changes they make,
  will be local.
* A local clone will try to hard link to the object directory.  I think
  it will copy them if it fails, or you can force that with "git clone
  --no-hardlinks".  For a more space-saving version, try "git clone
  -s", which will make a sort of soft link to the upstream repository.
  It's a git concept, so repacking upstream won't do any harm, but you
  Must Not delete objects from the upstream repository or you'll create
  dangling references in the downstream.
* If using the objects on the DFS mount turns out to be slow, you can
  just do the initial clone with --no-hardlinks.  Then the developers'
  day-to-day work is all local.

Indeed, you could easily do everything via DFS.  Give everyone a personal
"public" repo to push to, which is read-only to everyone else, and let
the integrator pull from those.

> I understand that avoiding repacking for an extended period brings its
> own problems, so I guess I could live with a local repack followed by
> an rsync transfer to re-initial the shared remote, if this was
> warranted.

Normally, you do a generational garbage collection thing.  You repack the
current work frequently (which is fast to do, and to share, because
it's small), and the larger, slower, older packs less frequently.

Anyway, I hope this helps!

```

## Jon Seymour, 2011-04-13 04:57

Subject: Re: is hosting a read-mostly git repo on a distributed file system practical?
Message-ID: <BANLkTi=cE4=EC0C4MA5iac6_O-YMsW_SLg@mail.gmail.com>
URL: https://gitlist.dev/e/BANLkTi%3DcE4%3DEC0C4MA5iac6_O-YMsW_SLg%40mail.gmail.com
In-Reply-To: <20110413034715.15553.qmail@science.horizon.com>

```
On Wed, Apr 13, 2011 at 1:47 PM, George Spelvin <linux@horizon.com> wrote:

> I think the answers are yes, but I have to make a vouple of things clear:
> * You can *definitely* control repack behaviour.  .keep files are the
>  simplest way to prevent repacking.

Good.

> * Are you talking about hosting only a "bare" repository, or one with
>  the unpacked source tree as well?  If you try to run git commands on
>  a large network-mounted source tree, things can get more than a bit
>  sluggish; git recursively stats the whole tree fairly frequently.
>  (There are ways to precent that, notably core.ignoreStat, but they
>  make it less friendly.)

Bare. Developers use local disk for local repos and working tree.

> * You can clone from a repository mounted on the file system just as
>  easily as you can from a network server.  So there's no need to set
>  up a server if you find it onconvenient.

Are there advantages to using rsync for the initial clone? Will I get
better restartability in the case that the network is less than 100% reliable?

I do remember trying to use a DFS-file system in the past, before I understood
pack management properly and I seem to recall issues with network reliability.

> Indeed, you could easily do everything via DFS.  Give everyone a personal
> "public" repo to push to, which is read-only to everyone else, and let
> the integrator pull from those.
>

I'll probably use ssh-secured peer to peer for publishing purposes.
The main thing I want
the DFS-hosted repo for is to provide a single, always up, go-to point
for the shared tag set.

>
> Anyway, I hope this helps!
>

Yep, thank you.

jon.

```