git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Why does fast-import need to check the validity of idents? + Other ident adventures

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Feb 5, 2021, 15:25 UTC
Message-ID
<87k0rmcza4.fsf@evledraar.gmail.com>
In-Reply-To
<xmqq7dnpc610.fsf@gitster.c.googlers.com>
On Wed, Feb 03 2021, Junio C Hamano wrote:
Show 25 quoted lines
> "=?utf-8?B?w4Z2YXIgQXJuZmrDtnLDsA==?=" Bjarmason <avarab@gmail.com>
> writes:
>
>> But I was wondering about fast-import.c in particular. I think Elijah's
>> patch here is obviously good an incremental improvement. But stepping
>> back a bit: who cares about sort-of-fsck validation in fast-import.c
>> anyway?
>
> Those who want to notice and verify the procedure they used to
> produce the import data from the original before it is too late?
>
> I.e. data gets imported to Git, victory declared and then old SCM
> turned gets off---and only then the resulting imported repository is
> found not to pass fsck.
>
>> Shouldn't it just pretty much be importing data as-is, and then we could
>> document "if you don't trust it, run fsck afterwards"?
>
> If it is a small import, the distinction does not matter, but for a
> huge import, the procedure to produce the data is likely to be
> mechanical, so even after processing just a very small portion of
> early part of the datastream, systematic errors would be noticed
> before fast-import wastes importing too much garbage that need to be
> discarded after running such fsck.  So in that sense, I suspect that
> there is value in the early validation.

What I was fishing for here is that perhaps since fast-import was originally written this use-case of in-place conversion of primary data on a server might have become too obscure to care about, i.e. as opposed to doing a conversion locally and then "git push"-ing it to something that does transfer.fsckObjects.

Show 8 quoted lines
>> Or, if it's a use-case people actually care about, then I might see
>> about unifying some of these parser functions as part of a series I'm
>> preparing.
>
> I think allowing people to loosen particular checks for fast-import
> (or elsewhere for that matter) is a good idea, and you can do so
> more easily once the existing checking is migrated to your new
> scheme that shares code with the fsck machinery.

...allright, depending on how much of a hassle that is I might just add tests for the differences and leave this particular problem to someone else :)

Previous: Junio C Hamano
Message 12 of 12 in “fast-import: accept invalid timezones so we can import existing repos”
  1. fast-import: accept invalid timezones so we can import existing reposElijah Newren via GitGitGadget, May 28, 2020
  2. Jonathan NiederMay 28, 2020
  3. fast-import: add new --date-format=raw-permissive formatElijah Newren via GitGitGadget, May 28, 2020
  4. Junio C HamanoMay 28, 2020
  5. Jonathan NiederMay 29, 2020
  6. Jeff KingMay 29, 2020
  7. Junio C HamanoMay 29, 2020
  8. fast-import: add new --date-format=raw-permissive formatElijah Newren via GitGitGadget, May 30, 2020
  9. Jeff KingMay 30, 2020
  10. Why does fast-import need to check the validity of idents? + Other ident adventures=?utf-8?B?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason, Feb 3, 2021
  11. Junio C HamanoFeb 3, 2021
  12. Ævar Arnfjörð BjarmasonFeb 5, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.