Real Software Forums

The forum for Real Studio and other Real Software products.
[ REAL Software Website | Board Index ]
It is currently Sun Sep 24, 2017 10:57 am
xojo

All times are UTC - 5 hours




Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Stack overflow on Linux, not on Windows for same source code
PostPosted: Wed May 15, 2013 6:56 pm 
Offline

Joined: Wed May 15, 2013 6:07 pm
Posts: 13
Hello,
A heavily recursive application (an ASN.1 interpreter) was migrated from VB6 to RealBasic for the only purpose of providing Linux users the access to this application. And its RealBasic version works fine on Windows. However the Linux version of the same causes stack overflow for the same input data when the call stack is 52 levels deep. By placing breakpoints in VB6 and RB Windows version at the same point in source code where Linux version had shown stack overflow, it was noted there is no real problem in the code because both VB6 and RB Windows version of the application also show that call stack is 52 levels deep and these versions do not have stack overflow problem.

Real Software support has suggested to use threads and set the stack size as required.

Is there any solution possible to this problem please without modifying the source code?

Thanks,

Shahid.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Wed May 15, 2013 8:11 pm 
Offline
Real Software Engineer

Joined: Sat Dec 24, 2005 8:18 pm
Posts: 7858
Location: Canada, Alberta, Near Red Deer
Hard to say without actually seeing the code.
Often this is "the stack is too deep and so has consumed all available stack space in the process space".
But not always.
It can just be that each frame is enormous and you've run out of usable stack space.
But that's hard to know without seeing the code as diagnosing this remotely in the forums is really hard.

_________________
Norman Palardy (Real Software)


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Wed May 15, 2013 11:24 pm 
Offline

Joined: Fri Jan 06, 2006 3:21 pm
Posts: 12388
Location: Portland, OR USA
Doesn't Windows use a much larger stack space by default? Put your recursive code in a thread and set the stack size sufficiently large.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Thu May 16, 2013 7:28 am 
Offline

Joined: Wed May 15, 2013 6:07 pm
Posts: 13
Hello,
The implicit message in the initial post was that any source code in RB that is processing the same input data should have the same behaviour on Windows, Linux and MAC as long as it is not linked to the GUI and other OS specific areas. This application is decoding an input data stream that is encoded in ASN.1 BER format and then label that decoded data according to a given ASN.1 specification. So ideally this application should not cause stack overflow on Linux for same input data when the Windows version of the same application does not cause stack overflow when same input data is being decoded.

So in summary the observed problem does not seem to be linked to the structure of the source code or the use (or abuse) of the stack space. Instead this could be a problem of inconsistent behaviour of RB so that the Linux version of application based on same source code while processing the same input data suffers a stack overflow while the Windows version of the same application while processing the same input data has no such problem of stack overflow.

The above listed observation seems to show that the various stack sizes in Linux must be less than the corresponding stack sizes on Windows that the RB (or OS??) provides for its applications. If that is indeed the case then the question could be asked that "Is Real Software planning to make such stack sizes uniform across Windows, Linux and MAC and what is the time frame for the same please?".

Thanks,

Shahid.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Thu May 16, 2013 12:17 pm 
Offline

Joined: Fri Jan 06, 2006 3:21 pm
Posts: 12388
Location: Portland, OR USA
I'm not sure that's a reasonable expectation, but it's worth filing a feature request in Feedback.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Thu May 16, 2013 4:46 pm 
Offline
Real Software Engineer

Joined: Sat Dec 24, 2005 8:18 pm
Posts: 7858
Location: Canada, Alberta, Near Red Deer
Shahid wrote:
Hello,
The implicit message in the initial post was that any source code in RB that is processing the same input data should have the same behaviour on Windows, Linux and MAC as long as it is not linked to the GUI and other OS specific areas. This application is decoding an input data stream that is encoded in ASN.1 BER format and then label that decoded data according to a given ASN.1 specification. So ideally this application should not cause stack overflow on Linux for same input data when the Windows version of the same application does not cause stack overflow when same input data is being decoded.

In theory yes.
But even in C and other languages I've run into the "in practice" they don't.

Shahid wrote:
Is Real Software planning to make such stack sizes uniform across Windows, Linux and MAC

Not that I am aware of

Shahid wrote:
and what is the time frame for the same please?".

Since there's no feature request to do so that is indeterminant.
And even if there were a request it may not be prioritized to be worked on for some time
And we really dont ever say "oh this will be done by such and such a date"

_________________
Norman Palardy (Real Software)


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Thu May 16, 2013 6:06 pm 
Offline

Joined: Wed May 15, 2013 6:07 pm
Posts: 13
Hello,
Cannot complain, having used the trial version of RB on Windows to migrate more than 10 MBytes of VB6 code to RB in less than 50 days and more than 200 hours so that Linux users could also have access to the same application, only to find out that the stack sizes on Linux are most likely less than those on Windows in RB.

So it seems "Goodbye" to RB after long and “short” tumultuous 50 days.

Thanks to "Real Software" for the free trial version of RB,

Shahid.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Thu May 16, 2013 7:19 pm 
Offline
Real Software Engineer

Joined: Sat Dec 24, 2005 8:18 pm
Posts: 7858
Location: Canada, Alberta, Near Red Deer
Shahid wrote:
Hello,
Cannot complain, having used the trial version of RB on Windows to migrate more than 10 MBytes of VB6 code to RB in less than 50 days and more than 200 hours so that Linux users could also have access to the same application, only to find out that the stack sizes on Linux are most likely less than those on Windows in RB.

So it seems "Goodbye" to RB after long and “short” tumultuous 50 days.

Thanks to "Real Software" for the free trial version of RB,

Shahid.


SO you've used it successfully & done a lot of work and are ready to simply give up ?
I'm sure that given the opportunity to review your code (essentially under NDA of course) we could figure out whats happening.
And yes I can say that since I do work for Real.
It could easily be as simple as changing the stack size or something - but right now I'm guessing since I cant debug it to know for sure.

_________________
Norman Palardy (Real Software)


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Thu May 16, 2013 7:47 pm 
Offline

Joined: Wed Mar 17, 2010 10:33 am
Posts: 258
Location: Sydney, Australia
http://www.cs.nyu.edu/exact/core/doc/stackOverflow.txt
http://stackoverflow.com/questions/2656722/is-there-a-limit-of-stack-size-of-a-process-in-linux
http://www.linuxquestions.org/questions/programming-9/why-the-stack-size-limit-878108/


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Sat May 18, 2013 3:16 pm 
Offline

Joined: Wed May 15, 2013 6:07 pm
Posts: 13
Hello,
Thank you for the web links on small stack size in Linux and its possible solutions. However I do not know much Linux so I have not been able to increase the stack size. The machine I have limited access to (for testing the migrated VB6 application) is a HP Proliant x86 64 bit server running SuSE SLES 11 OS with 64 GB RAM. It is ironic that the Windows 7 PC has only 4GB RAM that I used to migrate the VB6 application to RB (and compile the Linux version on) for this server and I do not get any stack overflow on this Windows machine.

Could you help on how to set the stack size on SuSE SLES 11 OS please?

Thanks and Regards,

Shahid.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Sat May 18, 2013 3:53 pm 
Offline

Joined: Wed May 15, 2013 6:07 pm
Posts: 13
Hello,

There is no chance of giving up that easily when there is willingness on the part of "Real Software" (RS) to help with the reported issue of stack overflow. There may be no need to go as far as the NDA because the sample RB application listed below suffers a stack overflow when the call stack is 8402 levels deep on Windows. While on Linux the stack overflow for the same application occurs when the call stack is only 2083 levels deep.

There is very little doubt that if RS can help in finding a solution so that this sample application also survives till the call stack is 8402 levels deep (before the stack overflow) on Linux then the VB6 application in question shall also work fine on Linux when the same solution is applied there.

Independent of the reported issue being discussed here, RS could make sure that the various stack sizes in RB for Windows, Linux and MAC applications are same or even larger than the corresponding stack sizes that VB6 provides for its applications.

I do not know that who controls these stack sizes, RB, OS or the person writing the RB code? If such stack sizes are configurable by the RB application developer or the RB application end user and if there are references to these in RB documentation then please forward those web links.

Thanks and Regards,

Shahid.

Create Desktop App:

Add following code in Window1.Recursive method:

//
// Start of file.
//
If lngCount < 100000 Then
lngCount = lngCount + 1
Window1.tfCount.Text = Trim(Str( lngCount))
Window1.Refresh
Call Window1.Recursive
Else
// Return
End If
//
// End of file.
//

Add following property in Window1 :

lngCount As Int32 = 0

Add a Push Button named “pbGo” and text = “Go” in Window1 and add the following code in “Action” event:

//
// Start of file.
//
Call Window1.Recursive
//
// End of file.
//

Add a TextField named “tfCount” in Window1.

Start the application in RB and click on the “Go” button and wait till the “stack overflow” exception.


Last edited by Shahid on Sat May 18, 2013 4:19 pm, edited 1 time in total.

Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Sat May 18, 2013 3:56 pm 
Offline

Joined: Fri Jan 06, 2006 3:21 pm
Posts: 12388
Location: Portland, OR USA
Your best bet is to put your code into a thread. Then you can set the thread's stack size to whatever you want. Much easier than messing around with the OS.

Moving to a thread is very easy. Usually, it's simply a matter of calling your code from the thread's Run event instead of from a button (or wherever it is now). Note that anything called from thread.run is run in the thread's context regardless of where it is defined. This means that you don't have to move any code into the thread object. Just call if from there.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Tue May 21, 2013 3:37 pm 
Offline
Real Software Engineer

Joined: Fri Sep 30, 2005 8:28 am
Posts: 242
Location: Unfortunately not Canada
Tim suggested a good around (by using Threads), otherwise you could also programmatically set the stack size by using a declare to setrlimit.


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Wed May 22, 2013 9:51 am 
Offline

Joined: Wed May 15, 2013 6:07 pm
Posts: 13
Quote:
Tim suggested a good around (by using Threads), otherwise you could also programmatically set the stack size by using a declare to setrlimit.



Hello,

Thank you.

The sample application from previous post above dated "Sat May 18, 2013 8:53 pm" was updated with the following code to call the getrlimit and setrlimit functions on SuSE SLES 11 OS running on HP Proliant server. It was noted that the existing stack size was 8MBytes and this was increased to 64MBytes. However the sample application still had a stack overflow after 2082 Recursive calls even when the setrlimit reported success while setting stack size to 64MBytes. While in RB on Windows the same sample application has a stack overflow after 8402 Recursive calls.

Please see what mistake I may be making here.

Thanks and Regards,

Shahid.

Code below was added to the "Window1 Open Event" in the sample application from previous post above dated "Sat May 18, 2013 8:53 pm"

//
// Start of file.
//

#if TargetLinux
if (System.IsFunctionAvailable("getrlimit" , "libc") ) And (System.IsFunctionAvailable("setrlimit" , "libc") ) then
Soft Declare Function GetResLimit Lib "libc.so" Alias "getrlimit" (resource As Integer, rlim As Ptr) As Integer
Soft Declare Function SetResLimit Lib "libc.so" Alias "setrlimit" (resource As Integer, rlim As Ptr) As Integer
Dim RLIMIT_STACK As Integer = 3 // defined in 'resource.h'
Dim result As Integer
Dim StackSize As UInt32
Dim ReqStackSize As UInt32
//
// struct rlimit {
// rlimit_r rlim_cur; /* Soft limit */
// rlimit_t rlim_max; /* Hard limit (ceiling of rlim_cur) */
// };
//
Dim rlim As new MemoryBlock(8) // rlimit_t is of type unsigned integer, four bytes each
// 64 MB
ReqStackSize = 64 * 1024 * 1024
// 32 MB
//ReqStackSize = 32 * 1024 * 1024
result = GetResLimit(RLIMIT_STACK, rlim)
MsgBox( "result = " + Trim(Str( result)))
If result = 0 Then
StackSize = rlim.UInt32Value(0)
MsgBox( "Soft limit = " + Trim(Str( StackSize)))
MsgBox( "Hard limit = " + Trim(Str( rlim.UInt32Value(4))))
MsgBox( "Requested Stack Size = " + Trim(Str( ReqStackSize)))
If StackSize < ReqStackSize Then
rlim.UInt32Value(0) = ReqStackSize
result = SetResLimit(RLIMIT_STACK, rlim)
If result = 0 Then
result = GetResLimit(RLIMIT_STACK, rlim)
If result = 0 Then
StackSize = rlim.UInt32Value(0)
MsgBox( "Set Stack Size = " + Trim(Str( StackSize)))
Else
MsgBox( "Error: Reading of stack size failed.after setting stack size to " + Trim(Str(ReqStackSize )))
End If
Else
MsgBox( "Error: Setting of stack size to " + Trim(Str( ReqStackSize)) + " failed.")
End If
End If
Else
MsgBox( "Error: Reading of stack size failed.")
End If
else
MsgBox ("Can't find 'getrlimit' or 'setrlimit' in 'libc'")
end if
#endif

//
// End of file.
//


Top
 Profile  
Reply with quote  
 Post subject: Re: Stack overflow on Linux, not on Windows for same source
PostPosted: Wed May 22, 2013 12:16 pm 
Offline
Real Software Engineer

Joined: Sat Dec 24, 2005 8:18 pm
Posts: 7858
Location: Canada, Alberta, Near Red Deer
Another of our engineers also pointed out that often a recursive solution feels "natural" but may be better served by an iterative solution where stack limits do NOT come into play.

Have you possibly tried converting this recursive parser into an iterative one where YOU manage the stack of pushes, pops) instead of relying on stack frames to do that for you.

An example of such a technique would be something like a search of files on disk.
You might start with a function like

Function searchfiles( f as folderitem ) as FolderItem()

dim tmp() as folderitem

for i as integer = 0 to f.Count - 1
if f.Directory then
dim list() as folderitem = searchFiles( f.item(i) )
for j as integer = 0 to ubound(list)
tmp.append list(j)
next
elseif matchesCriteria( f.items(i) ) then
tmp.append f.items(i)
end
next
return tmp
end function


As you can see this IS recursive
But the following would behave mostly the same and many fewer recursive calls (thus avoiding the stack overflow)
(this may not behave identically as I'm just writing it off the top of my head in the forums)

Function searchfiles( f as folderitem ) as FolderItem()

dim tmp() as folderitem
dim workList() as folderItem

worklist.append f

while ubound(worklist) > -1
dim current as folderItem = workList(ubound(worklist))
worklist.remove ubound(worklist)
for i as integer = 0 to current.Count - 1
if current(i).Directory then
worklist.append current.item(i)
elseif matchesCriteria( current.item(i) ) then
tmp.append current.item(i)
end
next
return tmp
end function

_________________
Norman Palardy (Real Software)


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next

All times are UTC - 5 hours


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group