Karya, built on 2022-03-21T01:30:44 (patch 89d1651424c35e564138d93424a157ff87457245)
Safe HaskellSafe-Inferred



Run tests. This is meant to be invoked via a main module generated by Util.Test.GenerateRunTests.

Since tests are in IO and write to stdout, to parallelize I can't just run a bunch of threads. So the parallelism is a bit weird: when run with --jobs, it will fork that many subprocesses with --subprocess. Each one will wait for a test to run on its stdin, and the parent process will dole them out when each is done.



data Test Source #




data Flag Source #


Instances details
Show Flag Source # 
Instance details

Defined in Util.Test.RunTests


showsPrec :: Int -> Flag -> ShowS #

show :: Flag -> String #

showList :: [Flag] -> ShowS #

Eq Flag Source # 
Instance details

Defined in Util.Test.RunTests


(==) :: Flag -> Flag -> Bool #

(/=) :: Flag -> Flag -> Bool #

data Jobs Source #


NJobs !Int 


Instances details
Show Jobs Source # 
Instance details

Defined in Util.Test.RunTests


showsPrec :: Int -> Jobs -> ShowS #

show :: Jobs -> String #

showList :: [Jobs] -> ShowS #

Eq Jobs Source # 
Instance details

Defined in Util.Test.RunTests


(==) :: Jobs -> Jobs -> Bool #

(/=) :: Jobs -> Jobs -> Bool #

run :: [Test] -> IO.IO () Source #

Called by the generated main function.

runInSubprocess :: Test -> IO.IO () Source #

Isolate the test by running it in a subprocess. I'm not sure if this is necessary, but I believe at the time GUI-using tests would crash each other without it. Presumably they left some GUI state around that process exit will clean up.

parallel jobs

runParallel :: [IO.FilePath] -> [Test] -> IO.IO [String] Source #

Run tests in parallel, redirecting stdout and stderr to each output.

shortenBy :: [a] -> [b] -> [a] Source #

jobThread :: IO.FilePath -> Queue (Text, [Test]) -> IO.IO (Maybe.Maybe String) Source #

Pull tests off the queue and feed them to a single subprocess.

testsCompleteLine :: Text Source #

Signal to the caller that the current batch of tests are done.

run tests

matchingTests :: [String] -> [Test] -> [Test] Source #

Match all tests whose names match any regex, or if a test is an exact match, just that test.

isolate :: IO.IO a -> IO.IO a Source #

Try to save and restore any process level state in case the test messes with it. Currently this just restores CWD, but probably there is more than that. For actual isolation probably a subprocess is necessary.

catch :: Text -> IO.IO a -> IO.IO () Source #


newtype Queue a Source #

This is a simple channel which is written to once, and read from until empty.


Queue (MVar.MVar [a]) 

newQueue :: [a] -> IO.IO (Queue a) Source #

check output

clearDirectory :: IO.FilePath -> IO.IO () Source #

Empty the directory, but don't remove it entirely, in case it's /tmp or something.

extractStats Source #


:: Text.Lazy.Text 
-> ([Text], Int, Int, Int)

(failureContext, failures, checks, tests)

extractFailures Source #


:: [Text.Lazy.Text] 
-> [Maybe.Maybe Text]

Just context for a failure, Nothing for a success.

Collect lines before and after each failure for context.

I collect before because that's where debugging info about that test is likely to show up, and I collect after because the failure output may have multiple lines.

It can be confusing that I can get the failure lines of the previous test as context for the current one. To fix that I'd have to explicitly mark all lines of the failure, or put some ending marker afterwards. It's not hard but maybe not worth it.