From 67bbc0fa210361782a2525b7574b1b176158ac2d Mon Sep 17 00:00:00 2001 From: Andrew Johnson Date: Fri, 26 Mar 2021 18:26:51 -0500 Subject: [PATCH] Increase default timeout, improve notes --- documentation/RELEASE_NOTES.md | 34 +++++++++++++++++----------------- src/tools/makeTestfile.pl | 14 ++++++++------ 2 files changed, 25 insertions(+), 23 deletions(-) diff --git a/documentation/RELEASE_NOTES.md b/documentation/RELEASE_NOTES.md index 3a37da919..ad3995dea 100644 --- a/documentation/RELEASE_NOTES.md +++ b/documentation/RELEASE_NOTES.md @@ -108,30 +108,30 @@ that the variables referenced by output pointers are initialized. ### Timeouts for Unit Test Programs -Most unit test programs that are run by the EPICS build system using -`make runtests` or `make tapfiles` are now only allowed to run for a limited -time period. If a test program runs for more than the timeout period (by -default 5 minutes, 300 seconds) without exiting it will be killed after that -period has elapsed. The limit can be extended (or shortened) for individual -tests as necessary by adding a line to the Makefile which runs them, setting -the environment variable `EPICS_UNITTEST_TIMEOUT` to a value in seconds while -building the test (`.t`) script, like this: +The unit test programs that are run by the `make runtests` or `make tapfiles` +commands get executed by a `.t` wrapper script which is normally generated by +the EPICS `makeTestfile.pl` program. Those generated wrapper scripts now +impose a time-limit on the test program they execute, and will kill it if it +runs for longer than 500 seconds (8 minutes 20) without exiting. That +time-limit can be changed for any such test by modifying the Makefile which +creates and runs the `.t` wrapper script. + +Setting the environment variable `EPICS_UNITTEST_TIMEOUT` to the desired +number of seconds while the Makefile is generating the test script changes the +timeout in that script. For example: ``` TESTSCRIPTS_HOST += hourLongTest.t hourLongTest.t: export EPICS_UNITTEST_TIMEOUT=3600 ``` -The time limit only applies to tests that use `makeTestfile.pl` to generate a -test (`.t`) script that runs the actual test program. Tests which are provided -as a `.plt` script should implement their own timeout if they can hang while -running in a Continuous Integration system (the Base "netget" test now does -this in a way that works on Windows as well as Posix hosts). +When selecting such a timeout remember that different Continuous Integration +systems such as GitHub Actions and Appveyor run on processors with different +speeds, so allow enough head-room for slower systems to complete the test. -The same environment variable can be set while running the tests to override -the time limit given in the Makefile, but it cannot be configured to provide a -separate limit for each program when using `make runtests`. This feature may -be useful for debugging purposes. +Test programs written directly in Perl as a `.plt` script should implement a +similar timeout for themselves. The "netget" test in Base does this in a way +that works on Windows as well as Unix-like hosts. ----- diff --git a/src/tools/makeTestfile.pl b/src/tools/makeTestfile.pl index 2d8fd3f9c..455bd7f69 100644 --- a/src/tools/makeTestfile.pl +++ b/src/tools/makeTestfile.pl @@ -21,17 +21,19 @@ # target.t is the name of the Perl script to generate # executable is the name of the file the script runs +# Test programs that need more than 500 seconds to run should have the +# EPICS_UNITTEST_TIMEOUT environment variable set in their Makefile: +# longRunningTest.t: export EPICS_UNITTEST_TIMEOUT=3600 +# That embeds the timeout into the .t file. The timeout variable can also +# be set at runtime, which will override any compiled-in setting but the +# 'make runtests' command can't give a different timeout for each test. + use strict; use File::Basename; my $tool = basename($0); -# Test programs that need more than 5 minutes to run should have the -# EPICS_UNITTEST_TIMEOUT environment variable set in their Makefile: -# longRunningTest.t: export EPICS_UNITTEST_TIMEOUT=3600 -# The above embeds it into the .t file. It can also be set at runtime, -# which will then override that compiled-in setting (so not recommended). -my $timeout = $ENV{EPICS_UNITTEST_TIMEOUT} // 5*60; +my $timeout = $ENV{EPICS_UNITTEST_TIMEOUT} // 500; # 8 min 20 sec my ($TA, $HA, $target, $exe) = @ARGV; my $exec;