Waldek Hebisch wrote:
Still, if you have Windows machine handy (and enough disk space) you can run the compilation on Mac OS X and then run the resulting executables on Windows. Some time ago I modified the test script to use wine (or dosemu) to run the executables so I could test cross-compilers on i386/Linux. I had to skip tests which used custom "compile/run" script, but it was easy to handle the rest. On Mac OS X you could just copy the MingW executables to some safe place and later run them on Windows. There is a little work since you want to check for failed compilation on host. Also on target you still need the test script to set up input and check for correct output.
My change to 'test_run' was basically to call a hook (shell function) to run exectables (intead of invoking them directly). Maybe we should include such change in reased version? By default the testsuite would run the same as now, but (say) setting environment variable we could change the behaviour.
I think so. Of course, it should handle all tests (or say unsupported if it's really not easily possible for a few) -- e.g., call that hook from the few special compile/run scripts where necessary.
Given the number of tests, a few thoughts about efficiency might be in order, though.
Running each test over ssh (I assume that's the general case, since Wine/Dosemu, of course, works only in special cases -- same hardware, OS emulator available) might take some time. Depending on the CPU speed and authentication method, each ssh login may take a few seconds, and that for several thousands of tests. But then, several hours running time may just be acceptable ...
Running all the tests over a single ssh login might be faster, but probably more difficult to write (interacting with a remote shell, transfer of binaries).
The third way, compiling everything first, transferring the binaries and running them remotely, is also more difficult to arrange. Also it raises questions of executable size (unfortunately -- though strip and tar.bz2 seem to bring them down to ~50 KB per binary, about 200 MB in total ...) Also, tests that require a program to run remotely before the actual compilation (such as fjf77) would require several steps here and there. Fortunately, there are very few of them ...
Frank