git fetch --reference?

When setting up a working area for kernel integration for a new embedded target, I generally do a "git clone --reference" so that the new area has its own repository (and its own branch structure) but most of the blobs come from a local reference copy. But now that I'm integrating bits from several non-trivially divergent trees (mtd-2.6, netdev-2.6, linux-2.6.16.y, etc.), it would be nice to avoid re-downloading blobs for these additional remote branches, which are also available in the local reference copy. Is it feasible to implement "git fetch --reference" for this purpose? Or is there a better way to manage this sort of integration effort?

Cheers,