git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Drastic jump in the time required for the test suite

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Oct 19, 2016, 09:18 UTC
Message-ID
<alpine.DEB.2.20.1610191049040.3847@virtualbox>
Hi Junio,

I know you are a fan of testing things thoroughly in the test suite, but I have to say that it is getting out of hand, in particular due to our over-use of shell script idioms (which really only run fast on Linux, not a good idea for a portable software).

My builds of `pu` now time out, after running for 3h straight in the VM dedicated to perform the daily routine of building and testing the git.git branches in Git for Windows' SDK. For comparison, `next` passes build & tests in 2.6h. That is quite the jump.

Unfortunately, I cannot tell you precisely when this massive increase happened (we are talking about half an hour, after all), because there were build failures preventing the test from being run for the last 40 days (and my build job only retains the results for 7 days currently, anyway, in addition to the last successful/unsuccessful build).

What I can say for certain is that the version from 41 days ago (commit 7837d4a) ran for only 2.6h, just as `next` does today, and passed without failure.

Every single revision of `pu` since then has been broken in one way or another. Mostly already the build, so that the tests would not even start to run.

What I can also say for certain is that the version from yesterday (commit 4ef44ce) was the first one in a week that built successfully and hence reached the test phase, and it was the first version of `pu` ever to time out after running for 3h.

I will increase the time-out, of course, but we are walking into a dangerous territory here: the build & test requires such an incredibly long time now that the VM will start to take more than 24h to complete all 8 tasks (building & testing the `pu`, `next`, `master` and `maint` branches, as well as trying to rebase Git for Windows' patches onto each of them).

As those tasks are scheduled to run once every 24h, I will no longer be able to notify you and the mailing list in a timely manner, if at all, because the queue will clog up.

Maybe we should start optimizing the tests to become more useful again, by forcing them not to take such an insane amount of time?

Ciao, Dscho

Next: Junio C Hamano
Message 1 of 31 in “Drastic jump in the time required for the test suite”
  1. Johannes SchindelinOct 19, 2016
  2. Junio C HamanoOct 19, 2016
  3. Jeff KingOct 19, 2016
  4. Johannes SchindelinOct 20, 2016
  5. Jeff KingOct 20, 2016
  6. Stefan BellerOct 20, 2016
  7. Jeff KingOct 20, 2016
  8. Stefan BellerOct 20, 2016
  9. Jeff KingOct 20, 2016
  10. Jeff KingOct 20, 2016
  11. Johannes SixtOct 21, 2016
  12. Jeff KingOct 21, 2016
  13. Johannes SchindelinOct 20, 2016
  14. Jeff KingOct 20, 2016
  15. Stefan BellerOct 20, 2016
  16. Jeff KingOct 20, 2016
  17. Jeff KingOct 20, 2016
  18. Johannes SixtOct 20, 2016
  19. Jeff KingOct 20, 2016
  20. Dennis KaarsemakerOct 20, 2016
  21. Duy NguyenOct 21, 2016
  22. Junio C HamanoOct 20, 2016
  23. Matthieu MoyOct 20, 2016
  24. Duy NguyenOct 20, 2016
  25. René ScharfeOct 20, 2016
  26. Duy NguyenOct 21, 2016
  27. René ScharfeOct 21, 2016
  28. Matthieu MoyOct 21, 2016
  29. René ScharfeOct 21, 2016
  30. Eric WongOct 27, 2016
  31. Duy NguyenOct 28, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.