Finally got it.
In the end the problem wasn't in the list work at all. Way earlier in the program, I'd allocated memory for an array [1..x], then accessed it as [0..x]. On freeing the memory, I guess something funny was set up in the memory manager that came back to bite me way later.
So... is there a compiler option I could have set that would have told me about this illegal memory access?
well, now perhaps I can sleep. Toby
Toby Ewing wrote:
Finally got it.
In the end the problem wasn't in the list work at all. Way earlier in the program, I'd allocated memory for an array [1..x], then accessed it as [0..x]. On freeing the memory, I guess something funny was set up in the memory manager that came back to bite me way later.
So... is there a compiler option I could have set that would have told me about this illegal memory access?
Glad to hear it. Now I won't do any stewing :-) gpc's lack of runtime range checking bites again :-[ But I'll bet a lot of it was your fault, in failing to declare a suitable index type and then the array as
aindex = 1..maxarr; ... arr : array[aindex] of whatever;
I'm not sure if gpc would then detect assignment of an out of range constant.
I maintain that careful declaration of vars and types, using the closest possible limits, pays off in the end.
Hi, all
CBFalconer wrote:
Toby Ewing wrote:
[...]
In the end the problem wasn't in the list work at all. Way earlier in the program, I'd allocated memory for an array [1..x], then accessed it as [0..x]. On freeing the memory, I guess something funny was set up in the memory manager that came back to bite me way later.
[...]
Glad to hear it. Now I won't do any stewing :-) gpc's lack of runtime range checking bites again :-[ But I'll bet a lot of it was your fault, in failing to declare a suitable index type and then the array as
[...]
Absolutely. This program was the merger of two previous programs that individually worked fine. When I merged them, I failed to check types as carefully as I should have: "We used to think that if we knew one, we knew two, because one and one are two. We are finding that we must learn a great deal more about 'and.'" (Sir Arthur Eddington)
In retrospect, I should have taken a bigger view when stewing over this bug: "hmm, each part worked fine. What about the combination isn't working?" But I got so caught up in where the bug was manifesting itself, that I lost perspective (and several days :-( ). Thanks to all who made suggestions.
CBFalconer wrote:
I maintain that careful declaration of vars and types, using the closest possible limits, pays off in the end.
Also true, and I like to think I usually do this. At least pre-merging!
But for when I don't ;-), and in lieu of a complete range checking implementation, would it be easier to write a stand-alone program parser that would examine a program for potential range-checking errors?
I respect Frank's feeling that range checking is not the most urgent issue for GPC. Range checking is probably an easier issue to debug on one's own than many of the other common bugs, and few of us use GPC for numerical work. I'll take my lumps as a lesson learned on this one.
regards, Toby
Toby Ewing wrote:
But for when I don't ;-), and in lieu of a complete range checking implementation, would it be easier to write a stand-alone program parser that would examine a program for potential range-checking errors?
Unless someone has a brilliant idea to do with very easily, I don't think it would be easier. In some cases it's even impossible (when the bounds depend on variables' values, the "parser" would have to simulate the program's behaviour which can only be done for specific cases, cf. halting problem).
Frank
Toby Ewing wrote:
Finally got it.
In the end the problem wasn't in the list work at all. Way earlier in the program, I'd allocated memory for an array [1..x], then accessed it as [0..x]. On freeing the memory, I guess something funny was set up in the memory manager that came back to bite me way later.
So... is there a compiler option I could have set that would have told me about this illegal memory access?
_No array index checking_ in gpc up to now unfortunately
Franck,
You have said some time ago that check of array addresses was not so important ...
You said also in an other mail that you were not fond of numeric analysis ...
The two are closely related because this kind of lost of days of time is by far the most important for anybody who does numeric analysis programs...
Correcting this is in my opinion the most urgent thing to do after the 2.1 release.
Maurice
Maurice Lombardi wrote:
Toby Ewing wrote:
In the end the problem wasn't in the list work at all. Way earlier in the program, I'd allocated memory for an array [1..x], then accessed it as [0..x]. On freeing the memory, I guess something funny was set up in the memory manager that came back to bite me way later.
So... is there a compiler option I could have set that would have told me about this illegal memory access?
_No array index checking_ in gpc up to now unfortunately
Franck, You have said some time ago that check of array addresses was not so important ... You said also in an other mail that you were not fond of numeric analysis ...
Correcting this is in my opinion the most urgent thing to do after the 2.1 release.
Chiming in with a 'me too'. What is needed is a way to check expressions as they are built, and prior to storing or using to index. It should look something like "check(min,max):boolean" This can be applied to subranges and indices, in fact whereever the compiler has knowledge of extreme allowable values (this includes case indices). If the machine model is a stack machine the model for this is easy.
In the long run the efficiency aspects can be ameliorated by deciding where NOT to call the checking code. This is where the numeric analysis comes in.
For arrays one of the preliminary operations is always to subtract the lower bound, so this can be combined with the check operation. Or you can look at it as being a check of a single value after manipulating into something zero based, but I think the lack of generality there will bite sooner or later.
So, here we go again. Must have been more than 2 weeks since the last discussion about range checking ...
Maurice Lombardi wrote:
You have said some time ago that check of array addresses was not so important ...
Indeed, it's not so important *to me*. I've had many more trouble with pointer bugs than with indexing bugs. Maybe that's due to my habits (checking indexes more carefully when I write them due to problems in the past) or my kind of programming (probably more with pointers and less with arrays than others) or the fact that index problems can often be easier found with some tools (efence, e.g.) ...
You said also in an other mail that you were not fond of numeric analysis ...
The two are closely related because this kind of lost of days of time is by far the most important for anybody who does numeric analysis programs...
If they'd spent some of these days working on GPC, then GPC might be better in this area. As it's usual with free software, if you want a feature, you can implement it yourself, you can pay someone to do it, or you can just hope.
Correcting this is in my opinion the most urgent thing to do after the 2.1 release.
CBFalconer wrote:
Chiming in with a 'me too'. What is needed is a way to check expressions as they are built, and prior to storing or using to index. It should look something like "check(min,max):boolean" This can be applied to subranges and indices, in fact whereever the compiler has knowledge of extreme allowable values (this includes case indices). If the machine model is a stack machine the model for this is easy.
In the long run the efficiency aspects can be ameliorated by deciding where NOT to call the checking code. This is where the numeric analysis comes in.
For arrays one of the preliminary operations is always to subtract the lower bound, so this can be combined with the check operation. Or you can look at it as being a check of a single value after manipulating into something zero based, but I think the lack of generality there will bite sooner or later.
Sorry, but we've discussed all these issues before, so let me point out again (and hopefully for the last time):
- I know quite well which cases need checking, on the source level. But it's some work to find the places in the compiler.
- We don't want a function. Doing the comparison inline is easy, more efficient (and maybe even smaller WRT code size; I'm not sure and I don't care too much).
- We don't need to worry about low-level optimization. The backend does this perfectly well already.
It's really not like this is very difficult in theory (in the basic version which is probably enough for most), it's just some work to do it, find all cases, test everything, etc. Let's look of the net result of this part of the discussion:
For me: 10 minutes lost For you: I don't know how much time lost For GPC: nothing gained
Not very effective, don't you think? So please, don't start such discussions again unless you have any new suggestions or actually want to help (same goes for porting to gcc-3 and other frequently repeated wishes). I don't react to the number of me-too's I see.
If you really want to help and don't dare to touch the compiler, one thing would be to write some good test programs -- by that, I don't mean 2 programs that contain the obvious cases that everyone thinks about, but a set of programs (maybe on the order of 20-50 programs) that test all conceivable cases, correct and wrong ones, and that meet the criteria of the test suite ...
Frank
Frank Heckenbach wrote:
... snip ...
CBFalconer wrote:
Chiming in with a 'me too'. What is needed is a way to check expressions as they are built, and prior to storing or using to index. It should look something like "check(min,max):boolean" This can be applied to subranges and indices, in fact whereever the compiler has knowledge of extreme allowable values (this includes case indices). If the machine model is a stack machine the model for this is easy.
In the long run the efficiency aspects can be ameliorated by deciding where NOT to call the checking code. This is where the numeric analysis comes in.
For arrays one of the preliminary operations is always to subtract the lower bound, so this can be combined with the check operation. Or you can look at it as being a check of a single value after manipulating into something zero based, but I think the lack of generality there will bite sooner or later.
Sorry, but we've discussed all these issues before, so let me point out again (and hopefully for the last time):
I know quite well which cases need checking, on the source level. But it's some work to find the places in the compiler.
We don't want a function. Doing the comparison inline is easy, more efficient (and maybe even smaller WRT code size; I'm not sure and I don't care too much).
We don't need to worry about low-level optimization. The backend does this perfectly well already.
Please don't don't consider anything sent to the mail list a criticism, or as addressed to you in particular. I certainly consider it as something for public consumption. You are much more familiar with the inner workings than anyone else I know, but obviously are not going to do everything. Mutterings about the general problems can sit in the rear of the mind of whoever eventually does something, and may very well help to gel his or her thoughts.
The last thing anyone should do is roll up their sleeves and start typing in code without a general attack clear.
CBFalconer wrote:
Frank Heckenbach wrote:
Sorry, but we've discussed all these issues before [...]
Please don't don't consider anything sent to the mail list a criticism, or as addressed to you in particular.
I didn't consider it as criticism or directed to me alone. I was pointing out, however, that all this had been discussed before on the list, so it's nothing new to anyone.
The last thing anyone should do is roll up their sleeves and start typing in code without a general attack clear.
BTW, if anyone really wants to work on these issues, they might want to contact me. I could give them some hints as to where to start in the compiler sources.
Frank
-
CBFalconer -
Frank Heckenbach -
Maurice Lombardi -
Toby Ewing