Development Progress Report, week 60

Well, I’ve built a lot on top of last week’s successes. Splitting the save-files into multiple sub-files was a good move. Following the logical thread of that, the compiled story-file is now a set of sub-files as well; and I was able to do something pretty awesome with that.

So, the new compiled story-file contains separate sub-files for general data and configuration, actor definitions, actor-data/knowledge-representation, location definitions, scene metadata, and scene performances.

This massively simplifies the process of getting the story into the engine, compared to having all of that packed nose-to-tail.

Now we get to the cool part.

The cool part!

Each scene in the story consists of a meta-data block, and a performance.

You might remember this example from the week 41 update:

Scene 2
 Summary:Some sample performance
 role:1:name=Lord Polonius [polonius]
 role:2:name=Prince Hamlet [hamlet]
 location:*:Somewhere // Move actors to the same place
 polonius>"What do you read, my lord?"
 hamlet>"Words, words, words."
 polonius>"What is the matter, my lord?"
 hamlet>"Between who?"
 polonius>"I mean, the matter that you read, my lord."
 hamlet>"Slanders, sir: for the satirical rogue says here that old men have grey beards...."
 polonius:Though this be madness, yet there is method in't.
 polonius>"Will you walk out of the air, my lord?"
 hamlet>"Into my grave."
 polonius>"Indeed, that is out o' the air."
 polonius:How pregnant sometimes his replies are!
  :A happiness that often madness hits on, which reason and sanity could not so prosperously be delivered of.
  :I will leave him, and suddenly contrive the means of meeting between him and my daughter.
 polonius>"My honourable lord, I will most humbly take my leave of you."
 hamlet>"You cannot, sir, take from me any thing that I will more willingly part withal."
 hamlet>"Except my life, except my life, except my life."
 polonius>"Fare you well, my lord."
 hamlet:These tedious old fools!

There’s two distinct sections there. The performance, and all of the meta-information required to set it up. Performances themselves can also contain nested performances (in a data-structure sense – it’s how conditional blocks are implemented).

The interesting thing here, is that we don’t actually need the performance – the body of the scene – until it comes time to, well, start performing the scene. Until that time, just the header information will suffice us.

That’s good, because by comparison, the performances take up a whole lot more space. The header itself is crunched down during compilation. The one for this scene would take up less than twenty characters in its final form.

So, now we load scene performances on-demand. Last week’s time-savings on the tokenise() function made sure that there’s no human-perceptible delays in doing that (it takes about 4 milliseconds, on average, now). Better, we can unload that performance again when the scene is done!

Memory is saved all-around!

Quirk: The story file (but not the accompanying assets file, containing images, audio, etc) is loaded into memory during startup and stays there. Right now, that’s a couple megabytes of data. What we’re saving here is memory spent on duplicate copies of data. Right now, there are reasons that I can’t just mmap() the story-file, like the assets are, but I won’t go into that all right now.

This did cause me some headaches with restoring save-files.

Where we are up to in a performance is kept as an index stored in the performance class, which indicates which action needs to be executed next. In the event that the next/current action contains a nested performance of its own, it keeps its own index. And so on all of the way down.

It’s neat and tidy and has managed itself to-date.

The problem here is that I can be restoring a save-file with nested performances or even sub-scenes (scenes called from within another scene, in the manner of functions or subroutines). And when I do, the actual performance structures are not currently populated, thanks to that nifty on-demand loading.

Some scenes are complete, some are not started, and others may be in various states of completion.

I toyed with the idea of moving performance indices up to the scene level, but the issue here is that all that elegant code and encapsulation for performances has no real concept of its enclosing scene, nor of how deeply it might be nested. It seemed kind of intractable.

In the end, I was able to work around it by having the scene query/store performance indices after every action (using basically the same code that I use to marshal the data prior to serialisation into a save-file), and restoration of that data after a performance is loaded in (again, using code that already existed).

At first I was a little concerned that this would take too many cycles, but my concerns were unfounded. It boils down to a handful of arithmetic/dereference operations and copying one or more integers around. Not a big deal, as it turns out!

Currently memory usage is stable between 200MB and 300MB – and probably the majority of that are graphical and audio assets actually in use. Sure, it’ll get larger as the story-file does, but very slowly.

Polymorphic Widgets are neat!

So far, all my on-screen widgets have been hand-rolled, and I’ve made heavy use of inheritance for functionality. The class diagram for widgets currently looks like this:

I decided that I wanted a widget that scrolled multiple lines of text in the manner of film-credits.

So, I created a new class from CustomButton and called it LinesOfText. CustomButton displays a single line of text in a filled frame with custom background, border, and text colours. It can be used for display/formatting or as a clickable control.

Twenty lines of code later, LinesOfText was a widget that used CustomButton’s functionality to do all of the same things, but which displayed multiple lines of text instead of a single line.

Then I created CreditScroller from LinesOfText. Twelve more lines of code, and it scrolls that text, fading in the bottom line and fading out the top line. No rewriting code to do things we already do, since those functions are available on classes further up the tree. Epic code-reuse.

Telemetry Improvements

Builds with built-in telemetry didn’t perform so well. Telemetry was single-threaded and DNS lookups and connection-establishment could take a while. This has been solved with some well-placed threading, rubbed vigorously into the network code, then capped with a mutex to stop network threads from knife-fighting over data structures. All good.

What were the mistakes, though?

The primary one was a minor audio bug with background music. The audio system should avoid triggering a music-track if that music-track is already playing (we simply store the name of the current track and compare it to the name of any new track that is requested). If you did indeed want to “take it from the top”, you’d stop music playback, then start the new file.

Well, the one place I did that, I only got silence.

Oh, haha! Someone forgot to clear the current track name when music playback was stopped.

Guess who that was?


Annoyingly, this bug took longer to find than a lot of far more complicated ones have. I burned about 90 minutes on this before the penny dropped. I didn’t spend that long nutting out my issue with performance indices during the big performance-loading/unloading refit.

Something can be right in front of you. That doesn’t mean you’ll see it right away.

End of a year

This is the last update for 2016!

I looked back through my code to see if I could figure out just when I started this project. So far, my history log goes back to 7 June, 2015.

The commit message is “Switched source control over to git”.

I dimly remember now that I had it in Subversion before that.

Let’s see now …

Okay, here we go. First line of code. First commit. A week earlier, on 1 June, 2015.

I’ve not worked on the project every week since then. We’re only up to 60 weeks of development since then, but wow. It’s been a while.

Longer than I thought.

And with that, I’ll wish you all a good New Year, and hopefully see you in 2017.