Toby Ewing wrote:
I'm working with some large arrays (2^27 elements), and am about to run into memory limitations. Currently I'm using a bytecard for each element, but in expanding my program's capabilities, I want to add a second array of cardinal. This second cardinal needs to have a range larger than 2^16 (65535), but I don't have enough memory for a 2^32 cardinal. Sounds like 2^24 cardinals would be appropriate...
I looked into the range-specified integers in the manual. It says:
8.2.3.3 Integer Types with Specified Size
In some situations you will need an integer type of a well-defined size. For this purpose, GNU Pascal provides three families of signed and unsinged integer types. The type Integer (42) is guaranteed to have a precision of 42 bits. In a realistic context, you will most often give a power of two as the number of bits, and the machine you will need it on will support variables of that size. If this is the case, the specified precision will simultaneously be the amount of storage needed for variables of this type. <<<
So perhaps the 24-bit cardinal would work fine. My questions:
- will the compiler really allocate only 24 bits, or will it align the
array elements along 32-bit boundaries? 2) If the compiler does indeed make true 24-bit-aligned cardinals, how much speed will I lose in memory access?
If you use a packed array or record, it will allocate only 24 bits (*), but you get all the drawbacks of packed records/array, such as greater likelihood of bugs (simply because they're tested much less) and loss of speed -- I don't have any figures (it might not be as bad on IA32 as on RISC machines). If you have no choice for memory reasons, you can only hope for the best I guess. But if you have to decide between upgrading either CPU or memory, I suggest you do some thorough tests.
You might also want to check whether two separate arrays or an array of a record containing an 8 bit and a 24 bit number are faster. The former makes the address calculation in the byte array easier (i.e., trivial), but the latter may provide better localized access and therefore better caching.
(*) Or rather, it should. I just tried it, and it didn't work. [...] Found the problem. GPC was making a special test to avoid packing if there's "nothing to pack", i.e. the type to be packed is a whole number of bytes already. Unfortunately, that's wrong -- the size must also be a power of 2. For 8 and 16 bits, it's no problem, but 24 bits is indeed the first size where it fails (and then 40, 48, 56 bits). I'm fixing it now (fjf582.pas), so in the next release, it will really allocate only 24 bits.
However, there might be another problem. I think GPC does some offset computations on the bit level, i.e. on a 32 bit machine they'll overflow with a size >= 2^32 bits. And since there may be sign issues, it might even be at >= 2^31 bits while your array would be 1.5 * 2^31 bits (and the combined array I suggested would even be 2^32 bits). I haven't looked closely into these matters, but if it is a problem, you might be forced to, e.g., spearate the 24 number into an array of 16 bit numbers and another one of 8 bit numbers.
Frank