Re: [PATCH] fast-import: fix over-allocation of marks storage

From: Jeff King <peff@peff.net>
Date: Oct 16, 2020, 20:25 UTC
Message-ID: <20201016202520.GB3356073@coredump.intra.peff.net>
In-Reply-To: <20201016031834.GE490427@camp.crustytoothpaste.net>

On Fri, Oct 16, 2020 at 03:18:34AM +0000, brian m. carlson wrote:

Show 14 quoted lines

> > Personally I make a branch for almost every patch/series I submit, no
> > matter how trivial[1]. And then part of my daily ritual is seeing which
> > ones have been merged, and dropping them. You can use git-cherry for
> > that, though it's not 100% perfect (sometimes patches are munged as they
> > are applied). I use a combination of that and aggressively rebasing
> > patches forward (and eventually they rebase down into nothing when
> > they've been fully merged).
> 
> I'm really terrible at deleting data, so I have (exactly) 256 branches
> in my local repository.  Some of them are merged, and some are not.
> This would be a viable approach if I were better about deleting old
> series (and completing and sending in the prototypes I've built), or if
> I sent in series that were smaller so rebasing them were not so big of a
> time commitment.

That's pretty good. I only have about 80. :)

I hesitate to point anybody at my mass of it-works-for-me scripts for fear of making their lives worse, but certainly you (or anybody) is welcome to adopt them if you want to do the aggressive-rebase thing. You can see them at:

  https://github.com/peff/git/tree/meta

I generally do:

  git clone --single-branch -b meta https://github.com/peff/git Meta

in my git.git checkout. And then:

  Meta/merged

shows what has been merged to master or next. Running:

  Meta/rebase

will try to rebase everything on its upstream (I almost always point branch upstreams to origin/master, though the script also handles the case that branch X's upstream is a local branch Y). That catches cases where the patch-ids changed enough that git-cherry can't recognize them.

Ideally that shrinks merged series down to nothing through a combination of automatic and manual skipping. Of course that will often yield conflicts if later already-merged patches in the series touched the same lines of code. You can either "git rebase --skip" past such commits, or if you realize the whole giant series is merged, just "rebase --abort" and delete the branch manually.

Perhaps either those scripts or at least the techniques within them might help with your cleanup.

Sort-of orthogonal, I also use:

  Meta/private

to create a branch "private" that is a merge of all of my topics on top of next (minus ones that have "-wip" in the name). That's what I use for my daily-driver version of Git.

That actually helps somewhat with the rebase process because rerere learns about conflicts early during this step, and then later the rebase can reuse the results. It's not perfect though (it merges in alphabetical order, but the order in which topics graduate might be different, meaning we see a conflict in reverse order. Junio's Meta scripts take a specific order from a topics file; but he has the luxury of knowing that the same topics file is what will be used to actually graduate the topics for real).

Anyway. I hope that might perhaps help a little, but don't feel compelled to jump down the rabbit hole if you're happy with your own workflows. ;)

-Peff