At 2:05 +0200 12/7/05, Frank Heckenbach wrote:
a) there is not much wasted memory (ie, wasting SizeOf(Integer) is fine, wasting 200+ bytes for a short string is not so good).
The latter never happens AFAIK, unless you allocate a string of a fixed "maximum" size.
Which would be what you'd need to do if you wanted to allocate a block of memory containing multiple strings - or at least, that is what I'm trying to avoid while allocating a block of memory.
a) Memory managers do not always work well with vast numbers of small allocations.
Do you have details? Of course, it depends on waht you mean by "not always". There probably are some better and some worse, but do you know which systems have bad ones? Fixing the MM would be a much more general solution than a hand-made solution for Pascal, strings, if explicitly used ...
Memory manages vary from system to system, version to version. I know early Mac memory manages did not like large numbers of small pointers. Of course, that was a different processor and a different operating system to modern Macs. Memory managers are tuned very carefully, but they are very general purpose in nature, it is inevitable they will not be optimum for many particular tasks.
Of course, any solution should only be implemented after profiling demonstrates that the simple clean solution is an expensive bottleneck - I'd just like to know if there is a way to pack Schema strings together to have that as an an option. Then I can compare and contrast the expense.
You can, of course, always create your pointers whichever way you like, taking care of alignment and sizes yourself. To get the size, you can use `SizeOf'. However, since it requires a type, or expressions of this type, it's a little more tricky to use here. This is one solution, perhaps the easiest one (pass the length of the string to be created).
function SizeOfString (Length: Integer): SizeType; type t = String (Length); begin SizeOfString := SizeOf (t) end;
Ok, great. That's what I wanted to know.
I wrote up a test program (attached below). It allocates 10000 x 20 strings, ranging in size from 5 to 24 characters. On my Mac, the New one took about 50% more time to run than the single block allocation one. Caveats:
* The single block size is pre-calculated, to be fair it needs to be calculated based on the sum of the required sizes.
* On the flip side, nothing is ever released, and Disposing all the pointers would certainly take longer than disposing the single block.
* Perhaps most importantly, the New solution took a few minutes to write and run reliably. The Ptr based one took much longer to write and required several corrections to the code before it past the tests. Clearly it is far more complicated, convoluted, and error prone.
I fear I am still stuck in the old days of caring too much about memory and cycles. Mind you, even with a machine running several billion instructions per second, it is amazing how often I am waiting for it to finish some task, so perhaps these are still issues - just issues that should be carefully chosen...
Thanks muchly for the assistance though, I feel I am much more in command of Schemas and the way things work now (both the clean way and the dirty way), although I certainly have a lot of new GPC features to learn to use to simplify some of the crud left over in my code from 68k days! Peter.
program peterZ;
type UInt8 = Cardinal attribute( size = 8 ); UInt8Ptr = ^UInt8; UInt16 = Cardinal attribute( size = 16 ); UInt16Ptr = ^UInt16; UInt32 = Cardinal attribute( size = 32 ); UInt32Ptr = ^UInt32; UInt64 = Cardinal attribute( size = 64 ); UInt64Ptr = ^UInt64; String255 = String(255); String255Ptr = ^String255; StringPtr = ^String;
function TheTime: UInt64; var t: TimeStamp; begin GetTimeStamp( t ); return ((t.Hour * 60 + t.Minute) * 60 + t.Second) * 1000000 + t.Microsecond; end;
const kMaxStrings = 20; kRepeats = 10000; kTotalStrings = kMaxStrings * kRepeats;
var gTheStrings: array[1..kMaxStrings] of String255;
type StringArray = array[1..kTotalStrings] of StringPtr; StringArrayPtr = ^StringArray;
procedure InitTheStrings; var s: String255; i: Integer; begin s := 'test'; for i := 1 to kMaxStrings do begin s := s + Chr(Ord('A') + i - 1); gTheStrings[i] := s; end; end;
procedure TestNew( var sap: StringArrayPtr ); var i, j, p: Integer; begin New(sap); p := 1; for i := 1 to kRepeats do begin for j := 1 to kMaxStrings do begin New( sap^[p], Length( gTheStrings[j] ) ); sap^[p]^ := gTheStrings[j]; Inc(p); end; end; end;
procedure TestPtr( var sap: StringArrayPtr );
procedure StringNew( var storagep: StringPtr; var sp: StringPtr; const s: String ); type StringN = String (Length(s)); StringNPtr = ^StringN; var p: StringNPtr; siz: Integer; begin sp := StringPtr(storagep); p := StringNPtr(storagep); Initialize (p^); p^ := s; siz := (SizeOf(StringN)+3) and not 3; // Round up to four bytes Inc(UInt32(storagep),siz); end;
type StorageSpace = array[1..kTotalStrings*10] of UInt32; // 40 bytes per string var storage: ^StorageSpace; storagep: StringPtr; i, j, p: Integer; begin New(sap); New(storage); storagep := StringPtr(storage); p := 1; for i := 1 to kRepeats do begin for j := 1 to kMaxStrings do begin StringNew( storagep, sap^[p], gTheStrings[j] ); Inc(p); end; end; end;
function CheckSap( sap: StringArrayPtr ): String255; var good: Boolean; i, j, p: Integer; begin good := true; for i := 1 to kRepeats do begin for j := 1 to kMaxStrings do begin good := good and (sap^[(i-1)*kMaxStrings+j]^ = gTheStrings[j]); end; end; if good then begin return 'OK'; end else begin return 'failed'; end; end;
type TestProcType = procedure( var sap: StringArrayPtr );
procedure RunTest( const name: String; TestProc: TestProcType ); var start: UInt64; sap: StringArrayPtr; begin start := TheTime; TestProc( sap ); WriteLn( name, ': ', TheTime - start, ' ', CheckSap( sap ) ); Dispose( sap ); sap := nil; end;
var start: UInt64; sap: StringArrayPtr; begin WriteLn( TheTime ); InitTheStrings; RunTest( 'TestNew', TestNew ); RunTest( 'TestPtr', TestPtr ); RunTest( 'TestNew', TestNew ); RunTest( 'TestPtr', TestPtr ); end.