Auto-repo-repair

Hi folks,

suppose the following scenario:

I've broken some repo (missing objects), eg by messing something up w/ alternates, broken filesystem, or whatever. And I've got a bunch of remotes which (together) contain all of the lost objects.

Now I'd like to run some $magic_command which automatically fetches all the missing objects and so repair my local repo.

Is this already possible right now ?

thx

On Fri, Nov 16, 2012 at 06:51:45PM +0100, Enrico Weigelt wrote:

You can't reliably just grab the broken objects, because most transports don't support grabbing arbitrary objects (you can do it if you have shell access to a known-good repository, but it's not automated).

The simplest thing is usually to re-clone the known-good remotes, then copy the resulting packfiles into your original repository. You'll have duplicated objects until your next "gc", but the resulting repack should skip any corrupted objects and use the known-good ones.

-Peff

Hi,

can we introduce a new or extend existing transports to support that ?

cu

On Sat, Nov 17, 2012 at 4:21 AM, Enrico Weigelt <enrico.weigelt@vnc.biz> wrote:

How would the broken repository be sure of what it is missing to request it from the other side?

fsck will find missing objects.

On Sun, Nov 18, 2012 at 11:55 AM, Enrico Weigelt <enrico.weigelt@vnc.biz> wrote:

And what about the objects referred to by objects that are missing? Jeff's solution doesn't suffer from this recursivity problem.

Will be fetched after multiple iterations. We could even introduce some 'fsck --autorepair' mode, which triggers it to fetch any missing object from its remotes.

Maybe even introduce a concept of peer object stores, which (a bit like alternates) are asked for objects that arent locally availabe - that could be even a plain venti store.

cu

On Mon, Nov 19, 2012 at 5:35 PM, Enrico Weigelt <enrico.weigelt@vnc.biz> wrote:

I still think that it would make the most sense to do the following (if you insist on some sort of automated repair): (1) Fetch a "good" clone (or clones) into a temporary directory; (2) Cannibalize the objects from it (them); (3) Re-run git fsck and check for still-missing / unreachable items; (4) IF THE RESULT OF (3) IS ACCEPTABLE, run git gc to clean up the mess, discard / "merge" duplicate objects, and fix up the packfiles.

It is step (4) that requires the most user interaction. I could see building up a shell script that does all but (4) nearly automatically. None of this requires modifying Git itself.

Hi,

Well, I'd like to have some really automatic mode, which does everything ondemand.

Once we've got this not just for repair, but also to support quick partial clones that fetch more objects when required.

In fact, finally, I'd like to have some storage cloud where data automatically gets replicated to nodes which need the data, not just for VCS, but other purposes (backup, filestore, etc) too. But before inventing someting completely new (reinventing much of the wheel), I'd like to investigate whether git can be extended into this direction step by step.

cu