git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/5] handling 4GB .idx files

From
Thomas Braun <thomas.braun@virtuell-zuhause.de>
Date
Nov 15, 2020, 14:43 UTC
Message-ID
<323fd904-a7ee-061d-d846-5da5afbc88b2@virtuell-zuhause.de>
In-Reply-To
<20201113050631.GA744608@coredump.intra.peff.net>
On 13.11.2020 06:06, Jeff King wrote:
Show 6 quoted lines
> I recently ran into a case where Git could not read the pack it had
> produced via running "git repack". The culprit turned out to be an .idx
> file which crossed the 4GB barrier (in bytes, not number of objects).
> This series fixes the problems I saw, along with similar ones I couldn't
> trigger in practice, and protects the .idx loading code against integer
> overflows that would fool the size checks.

Would it be feasible to have a test case for this large index case? This should very certainly have an EXPENSIVE tag, or might even not yet work on windows. But hopefully someday I'll find some more time to push large object support on windows forward, and these kind of tests would really help then.

Previous: Johannes SchindelinNext: Jeff King
Message 8 of 19 in “handling 4GB .idx files”
  1. 0/5 handling 4GB .idx filesJeff King, Nov 13, 2020
  2. 1/5 compute pack .idx byte offsets using size_tJeff King, Nov 13, 2020
  3. 2/5 use size_t to store pack .idx byte offsetsJeff King, Nov 13, 2020
  4. 3/5 fsck: correctly compute checksums on idx files larger than 4GBJeff King, Nov 13, 2020
  5. 4/5 block-sha1: take a size_t length parameterJeff King, Nov 13, 2020
  6. 5/5 packfile: detect overflow in .idx file size checksJeff King, Nov 13, 2020
  7. Johannes SchindelinNov 13, 2020
  8. Thomas BraunNov 15, 2020
  9. Jeff KingNov 16, 2020
  10. Derrick StoleeNov 16, 2020
  11. Jeff KingNov 16, 2020
  12. Thomas BraunNov 30, 2020
  13. Jeff KingDec 1, 2020
  14. t7900's new expensive testJeff King, Dec 1, 2020
  15. Taylor BlauDec 1, 2020
  16. Derrick StoleeDec 1, 2020
  17. t7900: speed up expensive testJeff King, Dec 2, 2020
  18. Jeff KingDec 2, 2020
  19. Derrick StoleeDec 3, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.