Latest developments

Hi,

Here’s a quick update on some things I have been working on since the v1.0.0 release:

New fiber system.

The fiber system has always been a bit ropey – it simulates fibers with OS threads, using semaphores to sync them. This is robust, but has considerable overhead and I had always planned to change to ‘real’ fibers at some point – ie: when fibers are switched, all that needs to be done is the CPU registers (including stack pointer) have to be saved/restored to/from a chunk of memory somewhere. The overhead for this shouldn’t really be much more than for a plain function call.

I wasn’t expecting to have to tackle this for a while, but an interesting bug popped up that pushed it to the front the queue – when you tried to close an ‘image document’ in Ted2, it would abruptly crash.

This was happening because the ‘close document’ code in Ted2 actually runs on its own fiber and, being well behaved, attempts to discard the image. However, discarding an image ultimately results in a call to glDeleteBuffers – but GL context’s are bound to the current thread so, since a fiber is/was really a thread, there was no context bound when glDeleteBuffers was called and kaboom! This could have been worked around, but what it really meant that, with the old fiber system, NO rendering code could be called on a fiber.

Fortunately, there is an excellent boost ‘context’ library that provides support for fiber-like code. I eventually got this building on Windows but of course, being a boost/c++ library, it was pretty huge. But there was an interesting ‘asm’ directory in there that looked like it just might contain the *exact* asm code need to perform fiber switches. So I added asm support to mx2cc (ie: you can now #import .s and .asm files), played around with things for a couple of hours and it worked!

The new fiber system is therefore in and should be *much* faster. And as far as the OS is concerned, there’s only one thread so you can render (and do other stuff like use system requesters, which you also couldn’t before) on fibers no problem.

It appears to have the asm code for a ton of target ABIs too (and it’s likely better written than I could have managed on  my own) including android and ios. But alas, emscripten is likely to remain fiber-less for the foreseeable future.

OpenAL Audio

Money2 v.1.0.0 used SDL_Mixer for audio, but it quickly became apparent that was pretty limited and wasn’t going to cut it.

After looking at a few of the options, I decided to go with OpenAL (again). There are number of reasons for this: I’m pretty familiar with it, it’s relatively high level and easy to use, and it runs everywhere I currently care about (desktop, emscripten, android, ios). For Windows, I am using openl-soft, an open source ‘fork’ (I think) of the original OpenAL. It’s LGPL licensed, which means you can’t static link with it and have to ship an openal.dll with you app (or depend on the user having it installed) but I don’t consider this a biggy.

So the mojo.audio code has now been upgraded to openal so the Audio, Sound and Channel classes should now be fully functional – although currently only WAV and OGG files are supported.

I also had a play around with ‘dynamic streaming’ of audio in openal, something skidracer already had working in SDL – see the vsynth banana.

It’s currently highly experimental so the docs are @hidden, but the basic idea is you start a fiber and queue chunks of raw audio data to a channel on that fiber. But before you queue a chunk, you generally called WaitQueued() to ‘sync’ with the channel. Here’s a quick pseudo example:

There is, however, one major drawback to this: The system is effectively polled. I start a 20hz timer in the background to check when chunks become unqueued and, of course, 20hz ain’t very fast so it forces a certain amount of latency.

And there are OS events that can completely stop mx2 executing altogether – in particular, dragging a window around by the title bar seems to just halt EVERYTHING (not so window resizes).

I can up the timer rate, but probably not much past 60hz as once you hit the framerate (if SwapInterval=1), the app ends up being blocked by ‘SwapBuffers’. I have a slightly crazy idea here for moving SwapBuffers to a thread which I want to play with in future, but for now SwapBuffers sort of introduces a ‘minimal polling rate’.

Ideally, this stuff should be done with REAL threads (although of course that introduces its own problems) but even then, openal isn’t really designed for this as it has no mechanism to signal a semaphore or something when a buffer is unqueued. But with threads, you would at least be able to poll at any rate you wanted.

Native GLES20 rendering

This was something I kind of decided to have a go at on a whim, mainly because a number of people asked about it.

There is now a ‘GLWindow’ class (that extends Window) that can be used to render native GLES20. To do this, just extend GLWindow instead of Window and override the GLWindow.OnRenderGL() method.

You can also combine GLES20 and mojo rendering, ie: you can still override OnRender( canvas:Canvas ), do some mojo rendering, call Super.OnRender (which will ultimately call OnRenderGL), do some more mojo rendering etc.

Some caveats though:

• The choice of whether a depth/stencil buffer should be created must be made before the AppInstance is created. To achieve this, I have added an optional config:StringMap<String> parameter to AppInstance’s constructor – see glwindowtest demo for how to use it.
• I’ve added a ClearEnabled property to Window to allow you to disable the default automatic clearing of the Window, as you may need to do this youself in GL if you’re clearing depth buffers etc. This is probably useful for non-GL apps anyway, as you may want to clear a window with a fancy rect or something instead of just a solid color.
• If you want to do any native GL rendering outside of OnRenderGL, you must wrap the code in GLWindow.BeginGL(), GLWindow.EndGL() calls (which currently don’t nest but will). This applies even in the GLWindow contructor. Mojo does no checking here, so forgetting to do this may blow things up!
• When you’re rendering, you’re rendering to the entire window surface. Therefore, things like “letterbox” layout etc will not ‘just work’.
• Only GLES20 is currently supported so you will need to create shaders etc.

What’s next?

Android support is my current focus, and after a bit of experimentation I don’t think it’ll be too hard.

I had a play around with the android ndk a few days ago, and it turns out there’s a ‘tool-chain’ creator in there that generates what is effectively a ‘devtools’ style folder that contains the correct gcc-ish compilers for building native code on a particular host so I already have the basic ‘tools’ problem solved.

The ‘big picture’ idea here (and with ios) is to build an entire android app into a ‘dll’ instead of an ‘exe’, and simply copy it into a ‘working’ android studio project.

To deal with things like manifest settings, info.plist config etc, I think I’ve come up with a pretty good compromise between monkey1’s unmaintanable ‘hack everything to death’ approach, and having to use android studio etc to configure *everything* on the target project side. It would basically work like this:

• Monkey2 or yourself provide a ‘template’ Android Studio or XCode project. This would be ‘preloaded’ with the stuff needed for it to work, eg: with the googleplay-ads library already in the project. This saves me the huge amout of voodoo involved in working out how to ‘hack’ libraries into projects – something that takes like 2 clicks to do in Android Studio. Many libraries also involve making many other weird and wonderful tweaks to projects, which again is easy to do if you’re following the FAQ on how to add library XYZ to your project.
• There would be a monkey2_settings.json file in the template that basically contains a number of ‘name/default’ pairs. This could include things like “App Title”:”MyApp” etc. It would also contain an array of files or filetypes to ‘process’.
• When you create a ‘new android project’, ted2 pops up a list of settings from monkey2_settings.json. You then select “App Title” etc. or use the default.
• When you’re done, the entire template is copied to your new working project dir. All files of a filetype listed in the ‘process’ array from money2_settings.json will have any ${BLAH}’s contained in them replaced with the appropriate setting.
• Also, any *files* or  *directories* that have any ${BLAH}’s in them will also be renamed appropriately ( ${} may or may not work here – we’ll find something).
• Finally, if you want to change an app setting, the entire project will be ‘regenerated’, ie: the template will overwrite the existing working version, with ${}’s replaced again etc. Unless you’ve tinkered with the target project, this will ‘just work’ and you’ll be able to change all sort of stuff like app id’s etc that you can’t in monkey1. If you *have* tinkered with the target project, you’ll need to make a new (reusable!) template.
• There may also have to be a monkey2_build.json file that specifies where the output exe or dll goes, where assets go, where any #imported dlls go etc.

This is IMO probably the sanest approach to take and what I’m aiming for right now re: ‘target projects’.

Money!

As some of you have no doubt worked out, mx2 is not generating a whole bunch of income for me at the moment.

Fotunately, it’s not as bad as it probably looks. There have been several very generous one off donations, and some people have chosen to donate regularly via paypal instead of patreon, so the pateron page at least doesn’t tell the whole story. If it did, things would be pretty grim right now!

I think ultimately, if it ends up working (ie: I can keep doing it fulltime), this is likely to be how it will work – ie: there wont be a single source of income like patreon, but multiple sources of income combining to provide me with something approaching a livable wage.

But things aren’t quite there yet, so if you have been using or are interested in mx2 but haven’t dontated yet, please consider doing so.

And spread the word!

Bye,
Mark