| Safe Haskell | Safe-Inferred |
|---|
Util.Test.RunTests
Description
Run tests. This is meant to be invoked via a main module generated by Util.Test.GenerateRunTests.
Since tests are in IO and write to stdout, to parallelize I can't just run a
bunch of threads. So the parallelism is a bit weird: when run with
--jobs, it will fork that many subprocesses with --subprocess. Each
one will wait for a test to run on its stdin, and the parent process will
dole them out when each is done.
Synopsis
- data Test = Test {}
- testModuleMeta :: Test -> Testing.ModuleMeta
- testName :: Test -> Text
- metaPrefix :: Text
- data Flag
- = CheckOutput
- | ClearDirs
- | Help
- | Interactive
- | Jobs !Jobs
- | List
- | Output !IO.FilePath
- | Subprocess
- data Jobs
- options :: [GetOpt.OptDescr Flag]
- run :: [Test] -> IO.IO ()
- quitWithUsage :: [String] -> IO.IO a
- runTests :: [Test] -> [Flag] -> [String] -> IO.IO Bool
- getJobs :: Jobs -> IO.IO Int
- runOutput :: IO.FilePath -> Int -> [Test] -> Bool -> IO.IO Bool
- runInSubprocess :: Test -> IO.IO ()
- runParallel :: [IO.FilePath] -> [Test] -> IO.IO [String]
- shortenBy :: [a] -> [b] -> [a]
- jobThread :: IO.FilePath -> Queue (Text, [Test]) -> IO.IO (Maybe.Maybe String)
- subprocess :: [Test] -> IO.IO ()
- testsCompleteLine :: Text
- matchingTests :: [String] -> [Test] -> [Test]
- runTest :: Test -> IO.IO ()
- isolate :: IO.IO a -> IO.IO a
- catch :: Text -> IO.IO a -> IO.IO ()
- newtype Queue a = Queue (MVar.MVar [a])
- newQueue :: [a] -> IO.IO (Queue a)
- takeQueue :: Queue a -> IO.IO (Maybe.Maybe a)
- clearDirectory :: IO.FilePath -> IO.IO ()
- checkOutputs :: [IO.FilePath] -> IO.IO Bool
- readFileEmpty :: IO.FilePath -> IO.IO Text.Lazy.Text
- extractStats :: Text.Lazy.Text -> ([Text], Int, Int, Int)
- extractFailures :: [Text.Lazy.Text] -> [Maybe.Maybe Text]
Documentation
Constructors
| Test | |
Fields
| |
metaPrefix :: Text Source #
Constructors
| CheckOutput | |
| ClearDirs | |
| Help | |
| Interactive | |
| Jobs !Jobs | |
| List | |
| Output !IO.FilePath | |
| Subprocess |
options :: [GetOpt.OptDescr Flag] Source #
quitWithUsage :: [String] -> IO.IO a Source #
runInSubprocess :: Test -> IO.IO () Source #
Isolate the test by running it in a subprocess. I'm not sure if this is necessary, but I believe at the time GUI-using tests would crash each other without it. Presumably they left some GUI state around that process exit will clean up.
parallel jobs
runParallel :: [IO.FilePath] -> [Test] -> IO.IO [String] Source #
Run tests in parallel, redirecting stdout and stderr to each output.
jobThread :: IO.FilePath -> Queue (Text, [Test]) -> IO.IO (Maybe.Maybe String) Source #
Pull tests off the queue and feed them to a single subprocess.
subprocess :: [Test] -> IO.IO () Source #
testsCompleteLine :: Text Source #
Signal to the caller that the current batch of tests are done.
run tests
matchingTests :: [String] -> [Test] -> [Test] Source #
Match all tests whose names match any regex, or if a test is an exact match, just that test.
isolate :: IO.IO a -> IO.IO a Source #
Try to save and restore any process level state in case the test messes with it. Currently this just restores CWD, but probably there is more than that. For actual isolation probably a subprocess is necessary.
queue
This is a simple channel which is written to once, and read from until empty.
check output
clearDirectory :: IO.FilePath -> IO.IO () Source #
Empty the directory, but don't remove it entirely, in case it's /tmp or something.
checkOutputs :: [IO.FilePath] -> IO.IO Bool Source #
Arguments
| :: Text.Lazy.Text | |
| -> ([Text], Int, Int, Int) | (failureContext, failures, checks, tests) |
Arguments
| :: [Text.Lazy.Text] | |
| -> [Maybe.Maybe Text] | Just context for a failure, Nothing for a success. |
Collect lines before and after each failure for context.
I collect before because that's where debugging info about that test is likely to show up, and I collect after because the failure output may have multiple lines.
It can be confusing that I can get the failure lines of the previous test as context for the current one. To fix that I'd have to explicitly mark all lines of the failure, or put some ending marker afterwards. It's not hard but maybe not worth it.