Re: RFC: git cat-file --follow-symlinks?

From: Jeff King <peff@peff.net>
Date: Apr 30, 2015, 19:10 UTC
Message-ID: <20150430191043.GA4461@peff.net>
In-Reply-To: <1430420422.22711.37.camel@ubuntu>

On Thu, Apr 30, 2015 at 12:00:22PM -0700, David Turner wrote:

Show 17 quoted lines

> > > Also, BUILD files are scattered throughout the tree, so the entire tree
> > > would still need to be traversed.  At present, our monorepo is not quite
> > > large enough for this to matter (a full ls-tree only takes me 0.6s), but
> > > it is growing.
> > 
> > But aren't you asking git to do that internally? I.e., it can limit the
> > traversal for a prefix-match, but it cannot do so for an arbitrary
> > filename. It has to open every tree. So the extra expense is really just
> > the I/O over the pipe. That's not optimal, but it is a constant factor
> > slowdown from what git would do internally.
> 
> No, I'm not trying to find all BUILD files -- only ones that are in the
> transitive dependency tree of the target I'm trying to sparsely check
> out. So if the target foo/bar/baz depends on morx/fleem, and morx/fleem
> depends on plugh/xyzzy, then I have to examine those three places only.
> I don't have to examine anything in the gibbberish/ subtree, for
> instance.

OK, let me see if I understand your use case by parroting it back.

You _don't_ want to feed git a "find all BUILD" pattern, which is good (because it doesn't work ;) ). You do want to feed it a set of raw paths to find, because you're going to discover the paths yourself at each step as you recurse through the dependency-chain of build files. You don't actually care about feeding those paths to "ls-tree" at all. You care only about the _content_ at each path (and will parse that content to see if you need to take a further recursive step).

So I think git out-of-the-box supports that pretty well (via cat-file). And your sticking point is that some of the paths may involve symlinks in the tree, so you want cat-file to answer "if I had checked this out and typed cat /some/path/to/BUILD, what content would I get". Which brings us back to the original symlink question.

Is that all accurate?

I'm not sure that helps with the "how to handle symlinks" discussion, but at least your goals make sense to me at this point.

-Peff