Avoiding Stack Overflow

If you create long and elaborated macros or use lots of lots of text you probably will run into a stack overflow error.

At first only two things to do come to mind:

 *clean up you html as much as possible (e.g. remove all comment)
  * raise the stack. (start high and work your way down) This is actually a bad advice, below is explained why.

This happens because the way the macro parsing works. It uses a reggex parser that starts by looking for the largest possible string that matches the regex, then backtracking to a shorter string (using recursion) if the first one ends up not working. This ends up requiring a huge amount of stack space in certain pathological strings when combined with the particular regexes being used. Regular expression require stack space based on the amount of text entered and how that text might match the regex. That's the whole point of backtracking -- the regex might match up to a certain point, so it saves its state on the stack and then checks the next piece.

(A search on user "Azrhei" and the word "backtrack" or "backtracking" will probably find the threads that explain this in great detail.)

MapTool creates and destroys many threads on the fly and all the time. Set your stack as low as possible as it's part of the overall address space available to Java. And the larger you make the heap limit, the more of the address space that can be consumed by heap and thus be unavailable for stack space. This is normally only a problem for the 32-bit Java. You'll know you've hit this problem when you get an exception that says that a new Thread couldn't be created and the description will say "Native code". (In other words, it's the C language code that implements the JVM that is having the problem.)

There wont be done anything about this for MT 1.3. We'll be moving to JS for 1.4. This should eliminate the regex stack overflow entirely as all macros will be JS. And that language uses a lexical parser and not a bastardized (but easier to implement) regex parser. The MTscript parser is great at doing what it was designed to do, but it has been pushed waaaaay beyond it's original vision!