Peter N Lewis wrote:
One thing I didn't see mentioned in the thread, but I discovered recently is that the routines like ReadStr/WriteStr are not thread safe. In particular they use a global variable (LastReadWriteStrFDR) for the file handle internally (to avoid re-allocating it each time), which is not MP thread safe. Also, they use an IOResult global variable (InOutRes variable).
That's right. And there are more global variables in the RTS, including user-visible ones such as `Input' and `Output', and internal ones such as `FDRList'. Though a few are only initialized once (e.g., by the special RTS command-line options) and never changed, so probably harmless. Others such as `ExpEpsReal' in `Ln1Plus' are initilaized on demand, so they can be shared, but initialization has to avoid races (it doesn't currently, and I'm not sure if just moving the `Inited := True' statement would be really safe -- it may need a mutex in the worst case).
So I think we should start with a list of RTS variables and classify them to find out how to deal with each. Did you start making such a list already perhaps?
To properly handle MP threading, the RTS would, at the least, require everything to have a returned error result, rather than using IOResult, which would be a big change (although this would be desirable for New as well).
This change would be incompatible with existing RTS routines,
And unfortunately also with existing Pascal standards (for `New') or dialects (for `IOResult').
although it could possible be accomplished by placing a thread safe layer underneath the thread-unsafe standard RTS API, for example (very loose pseudo code) :
procedure WriteStr( xxx ) var error: yyy; begin fh := GetReadWriteStrFDR; error := WriteStrSafe( fh, xxx ) InOutRes := error; conditionally exit with a runtime error end;
It would also likely require additions/changes to the way the compiler interacted with the RTS.
This way would not be BP compatible, since the raising of runtime errors must be done in the caller as it depends on the {$I+} setting (at the place of call). Therefore (and because BP allows direct access to InOutRes), it probably has to be a global variable. -- Or at least look like one. The compiler could rewrite read and write usage of it, but what about passing it by reference? BP allows this, and I'm not sure this isn't done in practice (ATM I'm not even sure WRT my own code, though I could check that).
If we look at C, they have a similar issue with `errno'. AFAIK, thread libraries rewrite it (via a macro) to something like `(foo^)' (in Pascal syntax) where foo is a thread-dependent function or expression that returns the address of a per-thread variable. This allows all kinds of usage including reference passing (address taking in C), so it might work with some effort.
Any way you slice it, it'd be a fair amount of work to be entirely thread safe.
I suppose so. It's probably obvious from previous mails that I'm not really a fan of multithreading -- so far, in every case others would have (or have) said requires threading, I solved it with either multiplexed I/O (poll, select) or separate processes (with separate data) -- which are not without problems, either, but to me they seemed easier to handle. I just mention this to say that I'm not going to spend much effort in this direction myself.
Anyway, before someone starts working on it, we should carefully discuss how to do it, and weigh the advantages and disadvantages.
One disadvantages I see with your approach is that it adds overhead, even to non-threaded programs, something I obviously don't like. I also wouldn't like requiring a thread library to start with. (I've had bad experience trying to even build some foreign programs that required particular thread libraries ... incompatible versions and what not.) This probably menas we'll need two versions of the RTS library, though, of course, we should be able to build them from the same source, using conditionals -- or even a compiler flag (we might need one for the compiler/RTS interface anyway, as you say, so perhaps we can test this flag with {$ifopt} in the RTS, but that's just a detail).
We should also, as far as possible, concentrate the differences in as few places as possible, even if this means using some macros or other ugly tricks. But first of all, the fewer variables we need to deal with, the easier it will become.
Of course, apart from variables, we have to look at non-thread-safe library routines. (I'm not sure which are -- what about malloc etc. for a start? Does libc make it thread-safe already?)
Frank