git add -p is slow for sparse checkout

Hi team,

I am currently on a sparse checkout repository, and it looks like the `git add—p` command is much slower than in a full checkout repository.

When run, the `git add—p` command expands to the full index and takes more time to start showing the patch.

Also, we have git trace2 enabled, and the timings of the subcommands gave us an indication that the `git apply—-cached` subcommand is taking way too long, with an average P50 of 3000ms against 300ms in the full checkout repository. The other notable spikes are in `git diff-files --color` (180ms vs 80ms) and `git diff-files --no-color` (180ms vs 80ms).

I am really interested in understanding why `git add -p` expands to a full index, which I believe is the issue with start-up regression. Also, would be great to understand the difference in performance of the subcommands.