Re: Round-tripping fast-export/import changes commit hashes

From: anatoly techtonik <techtonik@gmail.com>
Date: Aug 10, 2021, 15:51 UTC
Message-ID: <CAPkN8xLN_fKdbU8ugxLYJ1YeCJ8CxBWh+kdhAq1mR8hfAe-NAA@mail.gmail.com>
In-Reply-To: <CABPp-BH5RhHR-KhhumuhZGy2F4ypUBoqgAatY5MKkQsB46KM4g@mail.gmail.com>

On Mon, Aug 9, 2021 at 9:15 PM Elijah Newren <newren@gmail.com> wrote:

Show 5 quoted lines

>
> The author wasn't interested in implementing that
> suggestion (and it's a low priority for me that I may never get around
> to).  The series also wasn't pushed through and eventually was
> dropped.

What it takes to validate the commit signature? Isn't it the same as validating commit tag? Is it possible to merge at least the `--fast-export` part? The effect of roundtrip would be the same, but at least external tools would be able to detect signed commits and warn users.

> [1] https://lore.kernel.org/git/20210430232537.1131641-1-lukeshu@lukeshu.com/

> Yes, and I mentioned several other reasons why a round-trip from
> fast-export through fast-import cannot be relied upon to preserve
> object hashes.

Yes, I understand that. What would be the recommended way to detect which commits would change as a result of the round-trip? It will then be possible to warn users in `reposurgeon` `lint` command.

Show 10 quoted lines

> (3) fast-export works by looking for the relevant bits it knows how to
> export.  You'd have to redesign it to fully parse every bit of data in
> each object it looks at, throw errors if it didn't recognize any, and
> make sure it exports all the bits.  That might be difficult since it's
> hard to know how to future proof it.  How do you guarantee you've
> printed every field in a commit struct, when that struct might gain
> new fields in the future?  (This is especially challenging since
> fast-export/fast-import might not be considered core tools, or at
> least don't get as much attention as the "truly core" parts of git;
> see https://lore.kernel.org/git/xmqq36mxdnpz.fsf@gitster-ct.c.googlers.com/)

Looks like the only way to make it forward compatible is to introduce some kind of versioning and a validation schema like protobuf. Otherwise writing an importer and exporter for each and every thing that may encounter in a git stream may be unrealistic, yes.

Show 5 quoted lines

> > P.S. I am resurrecting the old thread, because my problem with editing
> > the history of the repository with an external tool still can not be solved.
>
> Sure it can, just use fast-export's --reference-excluded-parents
> option and don't export commits you know you won't need to change.

How does `--reference-excluded-parents` help to read signed commits?

`reposurgeon` needs all commits to select those that are needed by different criteria. It is hard to tell which commits are not important without reading and processing them first.

Show 16 quoted lines

> Or, if for some reason you are really set on exporting everything and
> then editing, then go ahead and create the full fast-export output,
> including with all your edits, and then post-process it manually
> before feeding to fast-import.  In particular, in the post-processing
> step find the commits that were problematic that you know won't be
> modified, such as your signed commit.  Then go edit that fast-export
> dump and (a) remove the dump of the no-longer-signed signed commit
> (because you don't want it), and (b) replace any references to the
> no-longer-signed-commit (e.g. "from :12") to instead use the hash of
> the actual original signed commit (e.g. "from
> d3d24b63446c7d06586eaa51764ff0c619113f09").  If you do that, then git
> fast-import will just build the new commits on the existing signed
> commit instead of on some new commit that is missing the signature.
> Technically, you can even skip step (a), as all it will do is produce
> an extra commit in your repository that isn't used and thus will be
> garbage collected later.

The problem is to detect problematic signed commits, because as I understand `fast-export` doesn't give any signs if commits were signed before the export.

-- 
anatoly t.