[pstaabp]

I think it's time to get this into the wild... at least as a draft. I've been using this for problems updated in the OPL.

This is a test script with a helper module.

• test-pg-problem.pl runs tests on a set of pg problems and outputs the results in various formats.

• ProblemParser.pm parses a pg problem into the source and metadata and then can be used to extract correct answers from a problem.

I have run this on a few thousand problems in the OPL. It skips any problems with non-Math Object answer checking. There are still a few issues that I haven't ironed out yet--some are listed below.

For testing, the script can extract the correct answers from a problem and then test that those answers are correct. Otherwise, if testing data is within the problem. I have converted some problems into a YAML block at the bottom of the file), then it will test against those answers. I was thinking this is helpful for testing different correct answers (MultiAnswer) or different format of answers. I have examples in a repository: https://github.com/pstaabp/OPL-next

Note: I have a script to take problems and convert them to this new YAML format. This will be submitted later, but there is some code related to this in this branch. I can remove this if needed.

There is still some work to do on this. A couple of known problems that I haven't figured out:

• If a formula has a fraction within it, there is generally an error of the form:

Can't locate object method "super" via package "context::Fraction::Parser::Value" at (eval 663) line 755.

• Contrib/Hope/Multi1/05-04-Diagonalization/Geom_diag_03.pg is giving errors I can't track down yet. It may be that it is in a scaffold.

[pstaabp]

I've removed the pg-critic and check-metadata options from the script.

This still has the parsing of the testing information in a YAML block, but if we decide to go with the mini-YAML block as in #1558, then I'll update to support that.