Karya, built on 2020-11-26T21:03:17 (patch 23b5be2d53a9e8e7d6136cda5aae2849abe5cded)
Safe HaskellNone

Util.Test.RunTests

Description

Run tests. This is meant to be invoked via a main module generated by Util.Test.GenerateRunTests.

Synopsis

Documentation

data Test Source #

Constructors

Test 

Fields

data Flag Source #

Instances

Instances details
Eq Flag # 
Instance details

Defined in Util.Test.RunTests

Methods

(==) :: Flag -> Flag -> Bool #

(/=) :: Flag -> Flag -> Bool #

Show Flag # 
Instance details

Defined in Util.Test.RunTests

Methods

showsPrec :: Int -> Flag -> ShowS #

show :: Flag -> String #

showList :: [Flag] -> ShowS #

data Jobs Source #

Constructors

Auto 
NJobs !Int 

Instances

Instances details
Eq Jobs # 
Instance details

Defined in Util.Test.RunTests

Methods

(==) :: Jobs -> Jobs -> Bool #

(/=) :: Jobs -> Jobs -> Bool #

Show Jobs # 
Instance details

Defined in Util.Test.RunTests

Methods

showsPrec :: Int -> Jobs -> ShowS #

show :: Jobs -> String #

showList :: [Jobs] -> ShowS #

run :: [Test] -> IO.IO () Source #

Called by the generated main function.

runInSubprocess :: Test -> IO.IO () Source #

Isolate the test by running it in a subprocess. I'm not sure if this is necessary, but I believe at the time GUI-using tests would crash each other without it. Presumably they left some GUI state around that process exit will clean up.

parallel jobs

runParallel :: [IO.FilePath] -> [Test] -> IO.IO [String] Source #

Run tests in parallel, redirecting stdout and stderr to each output.

jobThread :: IO.FilePath -> Queue (Text, [Test]) -> IO.IO (Maybe.Maybe String) Source #

Pull tests off the queue and feed them to a single subprocess.

testsCompleteLine :: Text Source #

Signal to the caller that the current batch of tests are done.

run tests

matchingTests :: [String] -> [Test] -> [Test] Source #

Match all tests whose names match any regex, or if a test is an exact match, just that test.

isolate :: IO.IO a -> IO.IO a Source #

Try to save and restore any process level state in case the test messes with it. Currently this just restores CWD, but probably there is more than that. For actual isolation probably a subprocess is necessary.

catch :: Text -> IO.IO a -> IO.IO () Source #

queue

newtype Queue a Source #

This is a simple channel which is written to once, and read from until empty.

Constructors

Queue (MVar.MVar [a]) 

newQueue :: [a] -> IO.IO (Queue a) Source #

check output

clearDirectory :: IO.FilePath -> IO.IO () Source #

Empty the directory, but don't remove it entirely, in case it's /tmp or something.

extractStats Source #

Arguments

:: Text.Lazy.Text 
-> ([Text], Int, Int, Int)

(failureContext, failures, checks, tests)

extractFailures Source #

Arguments

:: [Text.Lazy.Text] 
-> [Maybe.Maybe Text]

Just context for a failure, Nothing for a success.

Collect lines before and after each failure for context.

I collect before because that's where debugging info about that test is likely to show up, and I collect after because the failure output may have multiple lines.

It can be confusing that I can get the failure lines of the previous test as context for the current one. To fix that I'd have to explicitly mark all lines of the failure, or put some ending marker afterwards. It's not hard but maybe not worth it.