# Crash when pushing large binary file 5 messages from 2012-11-16 to 2012-11-20. Participants: Thomas Gay, Nguyen Thai Ngoc Duy. Thread: https://gitlist.dev/t/32125 ## Thomas Gay, 2012-11-16 05:44 Subject: Crash when pushing large binary file Message-ID: URL: https://gitlist.dev/e/CA%2BAoP-n07k0r11O6ShOT8jHiX%2BTL%2B%3Dp8sZoei%2BGi%2B9gG_YyJqA%40mail.gmail.com ``` Using Git 1.8 on Mac OS X 10.7.5. I just added a large binary file to my repo, and each time I try to push it, Git crashes. I've attached the crash log to this email and pasted the console output below. Counting objects: 27, done. Delta compression using up to 4 threads. Compressing objects: 100% (16/16), done. error: unpack-objects died of signal 11 | 76.91 MiB/s error: pack-objects died of signal 13 error: failed to push some refs to 'ssh://...' -Tom ``` ## Nguyen Thai Ngoc Duy, 2012-11-16 06:25 Subject: Re: Crash when pushing large binary file Message-ID: URL: https://gitlist.dev/e/CACsJy8AAd0odtLDiZ%2BA%2BMhX1En_G97gF2uoHjVeJvekznY3JSw%40mail.gmail.com In-Reply-To: ``` On Fri, Nov 16, 2012 at 12:44 PM, Thomas Gay wrote: > Using Git 1.8 on Mac OS X 10.7.5. I just added a large binary file to How large exactly? > my repo, and each time I try to push it, Git crashes. I've attached > the crash log to this email and pasted the console output below. > > Counting objects: 27, done. > Delta compression using up to 4 threads. > Compressing objects: 100% (16/16), done. > error: unpack-objects died of signal 11 | 76.91 MiB/s > error: pack-objects died of signal 13 > error: failed to push some refs to 'ssh://...' If you set receive.unpacklimit to 1 on the receiving end, does it still crash? -- Duy ``` ## Thomas Gay, 2012-11-16 06:54 Subject: Re: Crash when pushing large binary file Message-ID: URL: https://gitlist.dev/e/CA%2BAoP-keNcjGLtxbb9t7sggur1e%2BXmbMN5apVJfEdfhqU2-JGQ%40mail.gmail.com In-Reply-To: ``` Thanks for the quick reply! On Fri, Nov 16, 2012 at 3:25 PM, Nguyen Thai Ngoc Duy wrote: > On Fri, Nov 16, 2012 at 12:44 PM, Thomas Gay wrote: >> Using Git 1.8 on Mac OS X 10.7.5. I just added a large binary file to > > How large exactly? 2.46 GB > If you set receive.unpacklimit to 1 on the receiving end, does it still crash? Yes. The crash log looks the same too. -Tom ``` ## Nguyen Thai Ngoc Duy, 2012-11-16 10:11 Subject: Re: Crash when pushing large binary file Message-ID: URL: https://gitlist.dev/e/CACsJy8DsKc5_i6VKeYv69ikQXQPwEh3UOa5_bLaa4ss5vLxOng%40mail.gmail.com In-Reply-To: ``` On Fri, Nov 16, 2012 at 1:54 PM, Thomas Gay wrote: >> If you set receive.unpacklimit to 1 on the receiving end, does it still crash? > > Yes. The crash log looks the same too. If it still says "unpack-objects died of signal 11" then it was not done the right way. The receiving end can use either unpack-objects or index-pack for storing the objects. I know unpack-objects is not ready for large blobs (though I cannot explain your crash log, that's why I still need you to test it this way). I was hoping to force it use index-pack and see it still crashes. If it does, we have other problems than unpack-objects not being ready for large blobs. If it does not, I'd say it's a known issue with a known solution (I was planning on merging unpack-objects functionality back to index-pack). We can try again this way. index-pack will be used if the number of transfer objects exceeds 100 (by default). You are pusing 16 objects, which is why unpack-objects is used. We can try to push garbage to the other end to meet the 100 limit, then reset the branch at the other end later. You can run "git gc" early on the other end to clean up garbage, or it'll be done automatically at some point in future. Make sure there is no changes in index and worktree, or adjust you may want to change the last four commands slightly. mkdir tmp for i in `seq 200`;do echo $i > tmp/$i; git add $i; done git commit -m 'useless stuff' git push # should not crash again git reset --hard HEAD^ git push -- Duy ``` ## Nguyen Thai Ngoc Duy, 2012-11-20 11:37 Subject: Re: Crash when pushing large binary file Message-ID: URL: https://gitlist.dev/e/CACsJy8Ai%3DJK8c5GbjXBY2A4mtxSUUTzrTJt%3DpuxXvDD1fhFmFg%40mail.gmail.com In-Reply-To: ``` On Tue, Nov 20, 2012 at 8:50 AM, Thomas Gay wrote: > Sorry for the late reply; I've been out of the office for the past few days. > > On Fri, Nov 16, 2012 at 7:11 PM, Nguyen Thai Ngoc Duy wrote: >> If it still says "unpack-objects died of signal 11" then it was not >> done the right way. > > I upgraded Git on both ends and tried setting it again. index-pack was > used this time, but it still failed: > > Counting objects: 38, done. > Delta compression using up to 4 threads. > Compressing objects: 100% (26/26), done. > error: index-pack died of signal 11 GiB | 62.44 MiB/s > error: pack-objects died of signal 13 > error: failed to push some refs to '...' > > Would you still like me to try your other method? Interesting. No, if index-pack fails too the culprit is something else. Does the stack trace look the same (i.e. crashed in write_sha1_file_prepare)? I still don't know how it crashes there. It'd be great if you could run it under valgrind (not sure if valgrind runs in OS X). Try building with NO_OPENSSL too to avoid external sha1 issues. -- Duy ```