Re: Filter smudge for secret restoration: no disk access?

From: Kache Hit <kache.hit@gmail.com>
Date: Nov 25, 2025, 07:28 UTC
Message-ID: <DEHLKBB96BBI.3V74A5NGVTVZA@gmail.com>
In-Reply-To: <xmqqms4bw7f7.fsf@gitster.g>

On Mon Nov 24, 2025 at 11:35 AM PST, Junio C Hamano wrote:

Show 7 quoted lines

> I do not think it is a total dogmatic prohibition, but is a
> practical piece of advice to be prepared in a situation where the
> file %f does not exist on the disk in the working tree.  Also even
> when the file %f does exist, its contents would not match (because
> it was smudged when it was checked out, and the user may have
> further modified it) what in the tree of the commit you are
> switching out of.

You're right, it can be tricky as there are several cases to handle. I try covering this and other cases in the script's tests.

However, isn't properly handling different scenarios a separate issue? Simplying my concept to "ignoring" instead of "redacting":

 * Clean: ignore certain lines, preventing them from being committed
 * Smudge: don't overwrite working copy of ignored lines on checkout

Then the functionality becomes line-wise analogous to gitignore working on whole files. My local copy of gitignored `.env` isn't overwritten when I checkout. I'm looking for the same, just line-wise.

On Mon Nov 24, 2025 at 11:35 AM PST, Junio C Hamano wrote:

Show 6 quoted lines

> ... one of the things the checkout needs to do may be to remove the
> existing file from the working tree, and then create a file anew
> (probably in a temporary file) and move it to the final place, in
> which case, your smudge filter may be called during "create a file
> anew" phase, where the old file F or G may be missing from the working
> tree.

The old file being missing, being wholly removed right away, is exactly what I'm running into. If the working copy was kept around for `smudge`, I could achive a basic implementation of line-wise ignore/redact.

As-is, git's clean -> smudge filters can:
 * idempotent op -> no-op, e.g. identing or formatting
 * perfect mapping -> map back, e.g. git-lfs
 * add info -> remove info, e.g. expand RCS keyword -> unexpand

But not:
 * remove info -> restore info, e.g. ignoring lines, redacting

- Kache

PS

I've just found a case I'm not yet handling: at the end of `smudge()`, any unused secrets from the "previous working copy" that haven't been restored into the template would be lost. It is analogous to having local changes to a file at commit `X` and checking out `Y` where that file has been deleted. Git avoids overwriting local changes by aborting the checkout.