Change in direction...

I've been thinking off and on about what improvements need to be made to the EDMMAPI. The project is on hold, but when work resumes I plan to make a few changes.

In my original specs, there was a standalone screen object and I didn't really have much of an application object. Also in my original specs was to divide the API into parts, this way the developer can either partially exclude or completely exclude what they don't need. Prototyping the EDMMAPI in Win32 took the API in a different direction (in terms of modularity, not portability).

One of my goals designing the API is to have creation and destruction as antonyms. This way the API is hopefully easier to understand. If you create any kind of file, you close it when done. If you create an object which allocates memory, you destroy it when done. etc

I avoided multiple inheritence because methods with the same name cause conflicts. Also, multiple inheritence is not portable to other languages, such as Pascal.

I'm now leaning towards using multiple inheritence. Each object will be written in a way to avoid conflict. Method names will be unique to each object. Member variables will be named and structured in a way such that they can either be unique or shared between objects.

For languages such as Pascal, multiple inheritence will be internally implemented through the use of object variables.

Array-based list improvement...

The EDMMAPI lists, whether on disk or in memory, pack the data towards the beginning. So entry 0 is the first item in the list. When the list needs more space, the end of the list holds the reserved area.

The more items that are added, the bigger the moves become, the more sluggish adding items becomes.

So what I thought of was to increase the size of the list proportional to how many items the list contains. The reserved area would then be divided into two parts: one half at the beginning of the list, the other half at the end of the list.

When one reserved area or the other becomes full, the data is once again placed in the middle.

The result is that when an item is inserted, the amount of memory that needs to be moved is significantly reduced.

Project on hold...

I recently found out that Microsoft will no longer support inline assembly language starting with the Win64 platform. Instead, they will offer commonly used compiler intrinsics. I wouldn't doubt it if they drew the line at 64-bits since it's just starting to become mainstream. Once they support Win64 inline assembly and people start using it, they're locked into it.

What that means for me is that I will have to re-write a significant portion of the EDMMAPI source code on top of already having to special case Visual C++ 6 Pro, Visual Studio 2008 and (more than likely) beyond. While Visual C++ 6 Pro is not currently being used, I do have plans to backport to it for Windows '9x compatibility.

I do have alternatives, however, each one has their complexities and disadvantages. I started a new project, and need to focus on that for a little bit to determine what my next move is.

Win32 exceptions and the EDVM...

Looks like there's an elegant way to recover from crashes after all... I stumbled on a 1997 article, "A Crash Course on the Depths of Win32 Structured Exception Handling" by Matt Pietrek in which the smallest snippet of code had a sweet nibble of assembly language. I'm placing a snippet below as a quick reference...

//==================================================
// MYSEH - Matt Pietrek 1997
// Microsoft Systems Journal, January 1997
// FILE: MYSEH.CPP
// To compile: CL MYSEH.CPP
//==================================================
#define WIN32_LEAN_AND_MEAN
#include
#include

DWORD scratch;

EXCEPTION_DISPOSITION
__cdecl
_except_handler(
struct _EXCEPTION_RECORD *ExceptionRecord,
void * EstablisherFrame,
struct _CONTEXT *ContextRecord,
void * DispatcherContext )
{
unsigned i;

// Indicate that we made it to our exception handler
printf( "Hello from an exception handler\n" );

// Change EAX in the context record so that it points to someplace
// where we can successfully write
ContextRecord->Eax = (DWORD)&scratch;

// Tell the OS to restart the faulting instruction
return ExceptionContinueExecution;
}

int main()
{
DWORD handler = (DWORD)_except_handler;

__asm
{ // Build EXCEPTION_REGISTRATION record:
push handler // Address of handler function
push FS:[0] // Address of previous handler
mov FS:[0],ESP // Install new EXECEPTION_REGISTRATION
}

__asm
{
mov eax,0 // Zero out EAX
mov [eax], 1 // Write to EAX to deliberately cause a fault
}

printf( "After writing!\n" );

__asm
{ // Remove our EXECEPTION_REGISTRATION record
mov eax,[ESP] // Get pointer to previous record
mov FS:[0], EAX // Install previous record
add esp, 8 // Clean our EXECEPTION_REGISTRATION off stack
}

return 0;
}

SEH has been around since the days of OS/2. If future platforms continue to support SEH, then the ED VM will be able to recover gracefully from stack overflows, or unintended NULL pointer references.

If not, the assembly code to call will have to be wrapped within __try/__except blocks. __except(EXCEPTION_EXECUTE_HANDLER) results in catching the exception.

According to Microsoft's documentation on Structured Exception Handling, it works for C and C++ but there is a possibility that an object's destructor is not called under C++.

Apparently catch doesn't trap stack overflows. The best method to recover from seems to be to use a __try/__except block, and call _resetstkoflow() within __except. _resetstkoflow seems to be backward compatible down to Windows '95. While it's not guaranteed to fix the problem, it seems to give the application a good chance of recovering.

Binding externals...

My original plans were to declare a function in a script, and dispense IDs in the order they're declared. The developer then performs the binding with the dispensed ID.

The problem with assigning IDs in sequential order is that changing the order of bindings will lead to the wrong variable, function or method being bound. Also, in the case of importing definitions from a script library, indices will need to be re-assigned but the developer will not have a way to keep track of which binding is which.

I could resort to using a module ID, along with the binding ID, but changing the order of bindings will still be a problem.

My goals for binding are...
- Keep binding strings out of the application
- Provide the option to declare bindings in a language neutral configuration file
- Provide the option for a calling convention to ensure the external method or function is called properly
- Compile script specific binding syntax to EDVM syntax so that bindings can be shared between languages through EDVM libraries
- Link script bindings with application bindings through a developer specified ID so that they can be performed in any order
- Bind each function in the form of BindFunc( id, func )
- Bind each method in the form of BindFunc( id, obj.castfunc )
- Bind each variable in the form of BindVar( id, varptr, size )
- Bind each member variable in the form of BindVar( id, obj.varptr, size )
- Centralize bindings into the VM executor object

Because the EDVM will use assembly language for direct external calls, each supported platform needs certain types of assembly code. While not portable, the required assembly code will be minimal.

When the script calls an application function, assembly code is needed to copy the virtual stack to the real stack, to set up a self pointer for object methods, and to call the method or function directly. The alternative is to make the developer declare a callback which pops data off the virtual stack, then calls the function on the script's behalf. The alternative is portable, but imposes a lot of overhead in terms of required effort.

When the application calls a script function or method, assembly code is needed to copy the real stack on to the virtual stack and call a dispatcher (JIT generation allows parameters to be used directly). The alternative is to make the developer declare a call function which pushes parameters on to the virtual stack, and call a dispatcher. The alternative, once again, is portable but also imposes additional overhead in required effort.

Current thoughts on JIT compilation...

I originally wanted to load a VM byte code into a JIT compiled native code. My current plans for doing so are fairly solid. When 1.0 is ready for release, I plan on writing a VM with full EDMMAPI bindings. This way applications can be prototyped before any actual coding is done.

Since byte code can be executed X number of instructions at a time, threading can be simulated on platforms which do not support threading. Unless setjmp/longjmp can be used, JIT compiled code would eliminate that possibility.

With that said, I can see a use for JIT compilation when the script has complete control over the application, which is the case of the VM with full EDMMAPI bindings. I can also see a use for JIT compilation when a script has to perform a time consuming task the application depends on. But for small scripts, and for the sake of simulated threading, I don't think it's worth it.

Quick thoughts on memory management...

Ever since I first designed the EDMMAPI, I wanted a way to check how much memory was being used by the application in order to set the system requirements in the finished product. Eventually I also wanted a way to detect memory leaks.

While not a perfect solution, the best way to achieve this is to override the global new, delete, etc functions.

But first, I'll need to research how different platforms and different compilers allocate memory to ensure that the overrides don't decrease performance.

Another possibility is to write a self-sufficient source and header file which enable overrides when dropped in, and disable overrides when removed from the project. While this is not what I had in mind, at least it could work as a short term solution.