In message 199805300919.LAA00481@esmeralda.gerwinski.deyou write:
According to Jeffrey A Law:
Or you tweak the tests to put their output into a string, then have the test check the contents of the string for correctness and do the right thing re: exit/abort. [...]
Some tests are specifically testing whether the output to a string is really equivalent to output to a file. One would, in this example, have to read the contents of the file into another string and compare, but it is a lot of work to re-write all this.
I certainly understand. One possibility to avoid rewriting all that stuff would be a relatively simple extension to the torture framework that had it compare an output file with an expected output file.
That's how some of the libio/libstdc++ tests work, we'd just basically have to pick out the pieces we need. Not a major problem (IMHO).
Think of C programs that behave differently with `--traditional', `--ansi', or `--strict-ansi'. GNU Pascal has to comply to three major dialects (ISO-7185, ISO-10206, BP 7.0) with subdialects (with/without "extended syntax" or Boolean short-circuit operations or whatever), and they are becoming even more (ANSI Object Pascal, Delphi).
OK. Is there any way to annotate the testcase itself for the extra options that it needs (like a comment in the source)?
We could handle this similar to how the torture framework handles loops:
The framework looks at the source file for loop constructs, and if they're found it arranges for additional tests with loop unrolling options.
A similar mechanism could be used to annote the source to note what dialect of Pascal is used (and thus indicate special flags for the test).
One possible solution would be to have a subdirectory for each important combination of flags. Multiply this with the number of directories we must have anyway (run, compile, noncompile) and we are left with ~100 directories.
Yea, this gets out of hand! :-)
Move the existing GPC test suite
ftp://agnes.dida.physik.uni-essen.de/home/peter/gpc-test.tar.gzinto the GPC source distribution, replacing the obsolete tests in `test'. Take the stuff in `testsuite' as a starting point for a DejaGNU-based testsuite for GPC. Someone can move the tests from `test' into `testsuite' step by step. And we should rename those directories to reduce the confusion.
This is quite reasonable. And if someone wants some help on extending the torture framework to handle the stuff mentiond above I can help.
jeff