Karya, built on 2022-03-21T01:30:44 (patch 89d1651424c35e564138d93424a157ff87457245)
Safe HaskellSafe-Inferred

Util.Test.RunTests

Description

Run tests. This is meant to be invoked via a main module generated by Util.Test.GenerateRunTests.

Since tests are in IO and write to stdout, to parallelize I can't just run a bunch of threads. So the parallelism is a bit weird: when run with --jobs, it will fork that many subprocesses with --subprocess. Each one will wait for a test to run on its stdin, and the parent process will dole them out when each is done.

Synopsis

Documentation

data Test Source #

Constructors

Test 

Fields

data Flag Source #

Instances

Instances details
Show Flag Source # 
Instance details

Defined in Util.Test.RunTests

Methods

showsPrec :: Int -> Flag -> ShowS #

show :: Flag -> String #

showList :: [Flag] -> ShowS #

Eq Flag Source # 
Instance details

Defined in Util.Test.RunTests

Methods

(==) :: Flag -> Flag -> Bool #

(/=) :: Flag -> Flag -> Bool #

data Jobs Source #

Constructors

Auto 
NJobs !Int 

Instances

Instances details
Show Jobs Source # 
Instance details

Defined in Util.Test.RunTests

Methods

showsPrec :: Int -> Jobs -> ShowS #

show :: Jobs -> String #

showList :: [Jobs] -> ShowS #

Eq Jobs Source # 
Instance details

Defined in Util.Test.RunTests

Methods

(==) :: Jobs -> Jobs -> Bool #

(/=) :: Jobs -> Jobs -> Bool #

run :: [Test] -> IO.IO () Source #

Called by the generated main function.

runInSubprocess :: Test -> IO.IO () Source #

Isolate the test by running it in a subprocess. I'm not sure if this is necessary, but I believe at the time GUI-using tests would crash each other without it. Presumably they left some GUI state around that process exit will clean up.

parallel jobs

runParallel :: [IO.FilePath] -> [Test] -> IO.IO [String] Source #

Run tests in parallel, redirecting stdout and stderr to each output.

shortenBy :: [a] -> [b] -> [a] Source #

jobThread :: IO.FilePath -> Queue (Text, [Test]) -> IO.IO (Maybe.Maybe String) Source #

Pull tests off the queue and feed them to a single subprocess.

testsCompleteLine :: Text Source #

Signal to the caller that the current batch of tests are done.

run tests

matchingTests :: [String] -> [Test] -> [Test] Source #

Match all tests whose names match any regex, or if a test is an exact match, just that test.

isolate :: IO.IO a -> IO.IO a Source #

Try to save and restore any process level state in case the test messes with it. Currently this just restores CWD, but probably there is more than that. For actual isolation probably a subprocess is necessary.

catch :: Text -> IO.IO a -> IO.IO () Source #

queue

newtype Queue a Source #

This is a simple channel which is written to once, and read from until empty.

Constructors

Queue (MVar.MVar [a]) 

newQueue :: [a] -> IO.IO (Queue a) Source #

check output

clearDirectory :: IO.FilePath -> IO.IO () Source #

Empty the directory, but don't remove it entirely, in case it's /tmp or something.

extractStats Source #

Arguments

:: Text.Lazy.Text 
-> ([Text], Int, Int, Int)

(failureContext, failures, checks, tests)

extractFailures Source #

Arguments

:: [Text.Lazy.Text] 
-> [Maybe.Maybe Text]

Just context for a failure, Nothing for a success.

Collect lines before and after each failure for context.

I collect before because that's where debugging info about that test is likely to show up, and I collect after because the failure output may have multiple lines.

It can be confusing that I can get the failure lines of the previous test as context for the current one. To fix that I'd have to explicitly mark all lines of the failure, or put some ending marker afterwards. It's not hard but maybe not worth it.