Segfault in xdl_merge is back

So I've been able to reproduce the segfault that was earlier reported in xdl_merge. Unfortunately its in the repo that I can't publish. I plan to spend some time tomorrow evening to attempt to further debug the problem. I would certainly appreciate any advice. :-)

I can say its *not* related to 1510fea7 (or its fix 5caf9232 for that matter). Tonight I only had a few minutes to look at the issue but reverting 1510fea7/5caf9232 does not fix the segfault on Cygwin, even though 5caf9232 appears to have fixed the issue for the original reporter on Linux.

If I recall it correctly we were segfaulting on line 197 of xmerge.c:

    197         t1.ptr = (char *)xe1->xdf2.recs[m->i1]->ptr;

according to my particular case m->i1 == 70, but it looks like xdf2.recs isn't that large as index 70 is not a valid pointer.

I'm suspecting this is actually some sort of memory corruption in the heap (due to a bad malloc/free) as the bug seems to rear its head only based on the data we are allocating/have allocated.

If you look at what 1510fea7 would do on Linux the 1510fea7 bug would send us into the else case of diff_populate_filespec where we malloc the file data during decompression from the pack. Yet when we fixed it with 5caf9232 we started to mmap the working tree file, avoiding the malloc/free. This behavior, plus the fact that it happens no matter what for a particular merge on Cygwin (but not other merges), leads me to suspect heap corruption.

I may try to bisect this on Cygwin, but I may need to go all the way back to pre-xdl_merge() to get a working merge-recursive, and I may just find the bug pointing at the original merge-recursive code, or just find it pointing at a random commit like what happened with 1510fea7. So bisection may not really help out very much.

Has anyone run merge-recursive through Valgrind lately? I don't have a setup handy to run it through and see if we have any obvious errors.