Waldek Hebisch wrote:

...

? procedural parameter and local procedure problems (? trampolines ?)

the following test failures look the same:

• backtraceerr.pas
• fjf35.pas
• fjf464c.pas
• fjf464e.pas
• fproc.pas, same
• knuth1.pas, same
• nicola4d.pas, same
• nlgpp.pas
• nlgpp2.pas
• t6334-2.pas
• t6p6p3p4.pas

<snip>

Anything I can do to investigate this further and report it to gcc bugzilla ?

Basic precedure I use is: first debug the test program. There is added difficulty that one has to down to assembly level. Once the exact problem with generated code is known one looks why the compiler generated this code.

The long list starting with backtraceerr.pas looks like tests for nonlocal jumps (and related). We had problems with them on various platforms in the past. To demonstrate that the problems indeed are backend problems we have translated few of the tests to C. The tests should be now in GCC testsuite, but probably buried between thousends of other tests. I would suggest finding tests attached to past bug reports about nonlocal jumps and check if they work with the C compiler. If you have problem finding the tests I can search for them. If the tests work with the C compiler then we need to do extra work. If not we should report that to backend developers.

I believe trampolines are failing rather non-local goto's. The nonlocxxgoto.pas tests pass with --no-pic, but a simplified version of fjf35.pas fails.

program tramp;

procedure pp(procedure proc); begin proc end;

procedure global;

procedure loc; begin end;

begin pp(loc) end;

begin global end.

Assembly with --no-pic:

__p__M0_S0_Pp: pushl %ebp # movl %esp, %ebp #, subl $8, %esp #, movl 8(%ebp), %eax # Proc, Proc call *%eax # Proc leave ret _Loc.0: pushl %ebp # movl %esp, %ebp #, subl $4, %esp #, movl %ecx, -4(%ebp) #, leave ret __p__M0_S1_Global: pushl %ebp # movl %esp, %ebp #, subl $40, %esp #, leal -24(%ebp), %eax #, tmp61 addl $0, %eax #, tmp60 andl $-1, %eax #, tmp60 movl $_Loc.0, %ecx #, tmp63 leal 10(%eax), %edx #, tmp64 subl %edx, %ecx # tmp64, movl %ecx, %edx #, tmp62 movb $-71, (%eax) #, leal -8(%ebp), %ecx #, movl %ecx, 1(%eax) #, movb $-23, 5(%eax) #, movl %edx, 6(%eax) # tmp62, leal -24(%ebp), %eax #, tmp59 addl $0, %eax #, tmp58 andl $-1, %eax #, tmp58 movl %eax, (%esp) # tmp58, call __p__M0_S0_Pp # leave ret

This would also mean that http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22119 is wrong (or right for the wrong reasons). I will try the gcc testsuite, but I bet trampolines fail there also. I couldn't find anything related to darwin/x86 and trampolines on the gcc mailing list, so I think they didn't even bother.

Regards,

Adriaan van Os

Adriaan van Os wrote:

I believe trampolines are failing rather non-local goto's. The nonlocxxgoto.pas tests pass with --no-pic, but a simplified version of fjf35.pas fails.

program tramp;

<snip>

Assembly with --no-pic:

<snip>

How does it fail? Since the assembly contains nothing Darwin specific I tried it on Linux and for me it "works".

Waldek Hebisch wrote:

...

Adriaan van Os wrote:

...

I believe trampolines are failing rather non-local goto's. The nonlocxxgoto.pas tests pass with --no-pic, but a simplified version of fjf35.pas fails.

program tramp;

<snip> > Assembly with --no-pic: > <snip>

How does it fail? Since the assembly contains nothing Darwin specific I tried it on Linux and for me it "works".

It crashes with a bus error KERN_PROTECTION FAILURE (0x0002) at call *%eax. The eax register points to the stack, so I fear the stack is marked non-executable. Whether this is a problem in the kernel, gcc or the linker, I have to find out. There is a Darwin linker option -allow_stack_execute, but it causes a linker error (undefined symbols: _main), which is another thing to find out why ...

Wow, I think we have a workaround !

[Darwin:~/gpc/testgpc/tramp] adriaan% gp tramp.pas --no-pic -Wl,-stack_addr,c0000000 -Wl,-stack_size,4000000

This instructs the Darwin linker to create a segment __UNIXSTACK and apparently the stack segment has the correct priviliges set, because "tramp" no longer crashes !

[Darwin:~/gpc/testgpc/tramp] adriaan% ./tramp

Thanks for trying the assembly code on Linux.

Regards,

Adriaan van Os

Waldek Hebisch wrote:

Adriaan van Os wrote:

...

...

It crashes with a bus error KERN_PROTECTION FAILURE (0x0002) at call *%eax. The eax register points to the stack, so I fear the stack is marked non-executable. Whether this is a problem in the kernel, gcc or the linker, I have to find out. There is a Darwin linker option -allow_stack_execute, but it causes a linker error (undefined symbols: _main), which is another thing to find out why ...

Wow, I think we have a workaround !

[Darwin:~/gpc/testgpc/tramp] adriaan% gp tramp.pas --no-pic -Wl,-stack_addr,c0000000 -Wl,-stack_size,4000000

This instructs the Darwin linker to create a segment __UNIXSTACK and apparently the stack segment has the correct priviliges set, because "tramp" no longer crashes !

Correct way is to define ENABLE_EXECUTE_STACK in gcc-3.4.x/gcc/config/i386/darwin.h. The definition should be similar to definitions in gcc-3.4.x/gcc/config/sol2.h and gcc-3.4.x/gcc/config/netbsd.h

Yes, adding the netbsd definition to gcc-3.4.x/gcc/config/i386/darwin.h solves all trampoline problems.

/* Attempt to turn on execute permission for the stack. This may be used by INITIALIZE_TRAMPOLINE of the target needs it (that is, if the target machine can change execute permissions on a page).

There is no way to query the execute permission of the stack, so we always issue the mprotect() call.

Note that we go out of our way to use namespace-non-invasive calls here. Unfortunately, there is no libc-internal name for mprotect().

Also note that no errors should be emitted by this code; it is considered dangerous for library calls to send messages to stdout/stderr. */

#define ENABLE_EXECUTE_STACK \ extern void __enable_execute_stack (void *); \ void \ __enable_execute_stack (void *addr) \ { \ extern int mprotect (void *, size_t, int); \ extern int __sysctl (int *, unsigned int, void *, size_t *, \ void *, size_t); \ \ static int size; \ static long mask; \ \ char *page, *end; \ \ if (size == 0) \ { \ int mib[2]; \ size_t len; \ \ mib[0] = 6; /* CTL_HW */ \ mib[1] = 7; /* HW_PAGESIZE */ \ len = sizeof (size); \ (void) __sysctl (mib, 2, &size, &len, NULL, 0); \ mask = ~((long) size - 1); \ } \ \ page = (char *) (((long) addr) & mask); \ end = (char *) ((((long) (addr + TRAMPOLINE_SIZE)) & mask) + size) \ \ /* 7 == PROT_READ | PROT_WRITE | PROT_EXEC */ \ (void) mprotect (page, end - page, 7); \ }

New testsuite results:

[Darwin:gcc/p/test] adriaan% make EXTRA_PFLAGS="--no-pic -read_only_relocs suppress" rm -f *.dat *.o *.s *.i *.gpi *.gpd *.gpc core a.out stderr.out *.exe testmake.tmp dummy.c dummy.pas dummy.out diff_cr*.tmp fixcr fixcr.exe rm -f todo/a.out todo/*.exe todo/*.o todo/*.s todo/*.i todo/*.gpi todo/*.gpd todo/core GP= PC="gpc" PFLAGS=" --autobuild -g -O3 -W -Wall -Wno-unused --no-pic -read_only_relocs suppress " PFLAGS_NO_PATHS="-g -O3 -W -Wall -Wno-unused --no-pic -read_only_relocs suppress " SRCDIR="." TEST_MAKE_FLAG=test-make-flag "./test_run" "*.pas" | tee test_log | "./test_sum" -d Test Run By adriaan on 2005-11-21 20:13:06 Native configuration is i686-apple-darwin8 (Darwin.local)

=== gpc tests ===

Running target any Running testsuite ...

UNSUPPORTED: agettext2test.pas UNSUPPORTED: agettexttest.pas UNSUPPORTED: aregextest.pas UNSUPPORTED: fjf165a.pas UNSUPPORTED: gmptest.pas FAIL: systemtest.pas

=== gpc Summary ===

# of tests 4965 # of expected passes 4959 # of unexpected failures 1 # of unsupported tests 5

gpc version 20051104, based on gcc-3.4.4

This means we now have a stable compiler for non-position independent code. The FSF compiler for powerpc-apple-darwin has -mdynamic-no-pic, but for i686-apple-darwin that switch is currently only in the Apple branch. Maybe I can port Apple's changes for -mdyamic-no-pic to gcc-3.4.4.

Regards,

Adriaan van Os

On 6 Dec 2005, at 02:59, Adriaan van Os wrote:

Waldek Hebisch wrote:

...

Adriaan van Os wrote:

...

...

It crashes with a bus error KERN_PROTECTION FAILURE (0x0002) at call *%eax. The eax register points to the stack, so I fear the stack is marked non-executable. Whether this is a problem in the kernel, gcc or the linker, I have to find out. There is a Darwin linker option -allow_stack_execute, but it causes a linker error (undefined symbols: _main), which is another thing to find out why ...

Wow, I think we have a workaround !

[Darwin:~/gpc/testgpc/tramp] adriaan% gp tramp.pas --no-pic -Wl,-stack_addr,c0000000 -Wl,-stack_size,4000000

This instructs the Darwin linker to create a segment __UNIXSTACK and apparently the stack segment has the correct priviliges set, because "tramp" no longer crashes !

Correct way is to define ENABLE_EXECUTE_STACK in gcc-3.4.x/gcc/config/i386/darwin.h. The definition should be similar to definitions in gcc-3.4.x/gcc/config/sol2.h and gcc-3.4.x/gcc/ config/netbsd.h

I now read that Apple has gone completely mad <http:// lists.apple.com/archives/Xcode-users/2005/Nov/msg00402.html>. Bad news.

Adriaan van Os

Very, very silly to disable all Pascal and Ada programs! I suggest we all file bug reports. I don't think this is Apple going mad, I think it's a low level decision by someone who doesn't know what nested routines are about).

Peter N Lewis wrote:

Adriaan van Os wrote:

...

I now read that Apple has gone completely mad http://lists.apple.com/archives/Xcode-users/2005/Nov/msg00402.html. Bad news.

I agree entirely with their reasoning (the stack should not be executable, it makes far too many exploits possible),

I don't, they have apparently problems with CFM code, so instead of fixing those problems they choose the easy way. Why don't they forbid the execution of *any* code on the Macintosh ? That would be the ultimate safety precaution.

but their solution does indeed sound like madness. Surely it cannot be all that hard to support nested functions without stack based execution?

You can put the trampolining code in a stack-like array (see the last section of http://people.debian.org/~aaronl/Usenix88-lexic.pdf) but that part of memory then still must be made executable. The Darwin kernel forbidding to make any memory executable would be the end of interfacing with CFM code - but you never know what they decide in Cupertino. I will be glad when at some point in the future we all work with free open-source operating systems where the decisions are taken by sane people on sane grounds, after thorough and open discussion. At present, we are entirely dependent on what some empty headed manager in Cupertino or Redmond decides.

Note that trampolines are not needed for nested procedures as such - only when you pass them as actual procedural parameter.

Also note that trampolines are far from optimal, e.g. for the PowerPC the data and execution cache must be flushed. Trampolines are only there for compatibility with C (http://gcc.gnu.org/onlinedocs/gccint/Trampolines.html). A more elegant and efficient solution is given by Niklaus Wirth in his book on compiler architecture [1] by passing the static link pointer along with the address of the nested procedure. I proposed that solution for FPC.

Regards,

Adriaan van Os

[1] Niklaus Wirth, Grundlagen und Techniken des Compilerbaus, Addison Wesley 1996, ISBN 3-89319-931-4 (also translate in English)

Waldek Hebisch wrote:

I think they do not dare to disable `mprotect': after all dynamic linker have to modify jump tables for correct execution. And Java it too important to forbid executing freshly generated code.

I hope so. It would seem more like paranoia than anything else. After all, if some malicious code can call mprotect, it can most likely call any other system function it wants, so it's too late anyway ...

Do not know about PPC, but on most processors it is enough to flush a single cache line, which is moderate cost. In fact the `mprotect' syscall is likely to cost much more. However, if the trampoline stack is separate from normal stack then you can save most of the `mprotect' calls.

This might indeed an attractive solution in this case (not that I plan to implement it myself, though ;-), and probably few programs will ever need more than one page for it.

IIRC on AIX you do not need trampolines because all funtions are called via a descriptor which contains static link pointer. Wirth solution probably only considers procedure parameters. If you want also function pointers then there is a choice between thick pointers (which contain both the address and static link) and pointers to descriptors.

But what is more efficient depends very much on expected usage: if you have many function pointer but only limited number of functions then thick pointers require much more space then either trampolines or pointers to descriptors. Similarly when you pass function pointers/parameters: thick pointers are more expensive. Calling functions via trampolines is probably the most expensive way, but if you use trampolines than the const of calling normal functions is the lowest one.

If you assume that most functions are ordinary functions, and that trampolines are created infrequently and used slightly more frequently then created, then trampolines look like an optimal solution. And I think that such assumptions are in fact quite realistic.

FWIW, I agree.

But there's also a more practical point: If we want to interface to C or other functions that take plain function pointers (except perhaps on AIX as you say), we must provide such pointers. AFAICS, this requires something like trampolines.

Frank

Adriaan van Os wrote:

Waldek Hebisch wrote:

...

Correct way is to define ENABLE_EXECUTE_STACK in gcc-3.4.x/gcc/config/i386/darwin.h. The definition should be similar to definitions in gcc-3.4.x/gcc/config/sol2.h and gcc-3.4.x/gcc/config/netbsd.h

I now read that Apple has gone completely mad http://lists.apple.com/archives/Xcode-users/2005/Nov/msg00402.html. Bad news.

It is probably good time to get ENABLE_EXECUTE_STACK for i386-darwin into mainline FSF gcc (assuming it is not there -- I have not checked). Gcc is now in stage 1. Bad part of this is they want you run tests to check that there is no regressions and in stage 1 you may have problems bootstrapping the compiler. Also many tests are likely to fail, so one need to compare two runs. But there is also good part: they will not ask if it fixes a regression.