Re: git archive generates tar with malformed pax extended attribute

From: Jeff King <peff@peff.net>
Date: May 29, 2019, 01:17 UTC
Message-ID: <20190529011744.GA22340@sigill.intra.peff.net>
In-Reply-To: <5f9ac691-4cb1-a15d-ca44-63fb64994d59@web.de>

On Wed, May 29, 2019 at 01:34:32AM +0200, René Scharfe wrote:

Show 6 quoted lines

> It figures.
> 
> So something like the patch below?
> 
> Parsing trees with symlinks twice is not ideal, but keeps the set
> structure simple -- a standard oidset suffices.

If blobs comes after trees (and they usually do in a pack), you can do it in a single pass by marking the blob as a symlink target, and then when we actually see that blob's contents, marking it as either OK or problematic. And then the finish() step just correlates those with the tree.

It does require O(n) storage in the number of symlinked blobs, but also O(n) in the number of symlinked tree entries (number of trees with symlinks times the number of entries in each such tree, _even if they're the same entry/blob as another tree).

That makes it a lot worse than the existing gitmodules check. There we only care about finding the .gitmodules blobs. So even though you have a ton of trees that mention .gitmodules (basically every root tree), the the .gitmodules file itself doesn't change much. So we only end up with a small oidset (and a small worst case for looking at objects twice).

But here the problem is in the tree, not the blob. So we're not finding suspect blobs, but rather re-checking each tree. And no matter what we do (whether it's visiting the object again, or creating a set or mapping with the object names) is going to be linear there. And a repository with a symlink in the root tree is going to revisit or put in our mapping every single root tree.

TBH, I'm not sure this fsck check was worth it even without the implementation complexity.

> The global variables are ugly.  Moving them into struct fsck_option
> would be possible, but not much better, as they aren't really
> options.

Yeah, we use the name "context" elsewhere for this, which is a bit clearer. In a real object-oriented design, I guess people would make an "Fscker" object, and it would carry the options and context forward. That's basically what our "context" objects are.

> FSCK_MSG_MISSING_TREE_OBJECT has never been used before, it seems.

Yeah, this is leftover cruft from my gitmodules series. I double-checked the early iterations to be sure, and it is definitely the case. I didn't dig down to find out the reason it went away, but I think it is probably that the original version tried harder to find .gitmodules only in root trees, and later we decided to just complain about it in any tree.

-Peff