According to Jeffrey A Law:
Or you tweak the tests to put their output into a string, then have the test check the contents of the string for correctness and do the right thing re: exit/abort. [...]
Some tests are specifically testing whether the output to a string is really equivalent to output to a file. One would, in this example, have to read the contents of the file into another string and compare, but it is a lot of work to re-write all this.
We could probably enhance the testsuite to check for specific output of a test -- we already do that for stuff like libio/libstdc++.
That would simplify things a lot.
What kind of flags do you need to give to trigger the test?
Think of C programs that behave differently with `--traditional', `--ansi', or `--strict-ansi'. GNU Pascal has to comply to three major dialects (ISO-7185, ISO-10206, BP 7.0) with subdialects (with/without "extended syntax" or Boolean short-circuit operations or whatever), and they are becoming even more (ANSI Object Pascal, Delphi).
One possible solution would be to have a subdirectory for each important combination of flags. Multiply this with the number of directories we must have anyway (run, compile, noncompile) and we are left with ~100 directories.
THe torture framework is design to run through a set of common flags (usually optimization options) for all tests.
That's not enough. :-(
There are ways to pass specific flags to specific tests, but it's frowned upon in the torture framework.
I conclude that DejaGNU currently does not seem to be able to do all tests necessary for testing GNU Pascal. Due to the omnipresent lack of human ressources I see no possibility to really solve this problem in a reasonable time. Thus I suggest the following:
Move the existing GPC test suite
ftp://agnes.dida.physik.uni-essen.de/home/peter/gpc-test.tar.gz
into the GPC source distribution, replacing the obsolete tests in `test'. Take the stuff in `testsuite' as a starting point for a DejaGNU-based testsuite for GPC. Someone can move the tests from `test' into `testsuite' step by step. And we should rename those directories to reduce the confusion.
Peter