Re: Keep reflogs for deleted (remote tracking) branches?

From: Tao Klerks <tao@klerks.biz>
Date: Mar 14, 2022, 08:25 UTC
Message-ID: <CAPMMpoiDTprbf_9J3gY6WQwUVWfOTms6LVyJDYQUOcUp-42doA@mail.gmail.com>
In-Reply-To: <220308.868rtky4q8.gmgdl@evledraar.gmail.com>

On Tue, Mar 8, 2022 at 2:05 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

Show 26 quoted lines

>
>
> On Tue, Mar 08 2022, Han-Wen Nienhuys wrote:
>
> > On Tue, Mar 8, 2022 at 12:28 PM Tao Klerks <tao@klerks.biz> wrote:
> >> As far as I can tell, even "core.logAllRefUpdates=always" does *not*
> >> keep any reflog entries around, even temporarily (until reflog
> >> expiry), once a ref  is deleted - do I understand that correctly? Is
> >> this behavior intentional / reasoned, or just a consequence of the
> >> fact that it's *hard* to keep "managing" per-branch reflogs for
> >> branches that don't exist?
> >>
> >> I am planning a workaround using server hooks to "back up" refs that
> >> are being deleted from specific namespaces, in my specific case, and I
> >> imagine that a system like github keeps track of deleted stuff itself
> >> for a while, but I find this "per-ref reflog disappearance" behavior
> >> puzzling / out-of-character, so wanted to make sure I'm not missing
> >> something.
> >
> > I think this behavior is motivated by directory/file conflicts. If you
> > have a reflog file in refs/logs/foo, you can't create a reflog for
> > refs/foo/bar, because that would live in refs/logs/foo/bar
> >
> > At Google, we keep reflogs in a completely different storage system
> > altogether, which avoids this problem, and I wouldn't be surprised if
> > other large hosting providers do something similar.

This is interesting - so at google is the assumption that the storage system, whatever it looks like, *does* keep reflogs for deleted branches? Or at least backs up states that get force-pushed out of existence?

Show 17 quoted lines

>
> I once worked on a system where:
>
>  * References would be "archived", i.e. just a backup system that would
>    run "git fetch" without pruning.
>
>  * You were only allowed to push to either existing branches like
>    "master", or names with exactly one slash in them, e.g. "avar/topic",
>    not "avar/topic/nested", for that you'd need "avar/topic-nested" or
>    whatever.
>
> The second item neatly avoids D/F conflicts, at the cost of some
> grumbling from people who can't use their preferred branch name.
>
> And you can easily implement backups without that constraint by fetching
> refs/* to refs/YYYYMMDD-HHMMSS/* or whatever, and have some manual
> pruning process in place for those "secondly refs".

Ah right, backing up into another system - I guess we could...

>
> More generally I have not really run into this as a practical
> problem.

That's fair, nor have I - but I *have* come reasonably close: one person accidentally deletes a branch that someone else had prepared *without even realizing*, and the initial author is not available, and I only find out about it a few hours later. Dangling commit hunt, here we come. (the original author became available and re-pushed before it came to that)

Show 6 quoted lines

>
> Another way to solve a similar problem is to have
> pre-receive/post-receive hooks log attempted/successful pushes, which
> along with an appropriate "gc" policy will allow you to manually look up
> these older branches (or even to fetch them, if you publish the log and
> set uploadpack.allowAnySHA1InWant=true).

Yep, that's closer to my expected plan, thanks - my intent is to back up, on force-push and/or deletion, into a specific refspace with a cleanup policy, using a server hook. So after something is "deleted" (or force-pushed away), it can be easily recovered for a period of eg 3 months in that refspace, eg "refs/force-push-backups/YYYY-MM-DD-<BRANCHNAME>-<HASHPREFIX>".

My question is specifically about the, in my opinion, very surprising behavior of deleting reflogs along with deleted branches - I mainly provided the example use-case for context.