Development Progress Report, week 33

Finally! That lingering scene that I couldn’t seem to get any traction on is done, and I’ve hit a big narrative milestone!

I’ve looked back over things, and decided to wrap that as chapter one, and have started in on chapter two. I’m going to need a few diagrams (in at least five dimensions) to work out the structure of the next few scenes, mind.

Narrative word-count, has become likewise pan-dimensional. I can’t say with any certainty what the word-count actually is.

I’ve been pulling a whole bunch of space-saving tricks, which means I’ve shaved more than ten-thousand words off of the total word-count, without actually shortening the narrative by a single word (in fact, it has grown some).

I can approximate it at a quarter-million words, but all that can ever be is an approximation. How many words you get is different. A lot. More than that.

(For reference, the demo weighs in at ~125,000 fuzzy-quantum-words).

Memory

This week has been focused a lot on memory usage.

As the story and its assets have grown larger, I’ve become more concerned about the amount of memory the whole thing takes up. Initially, I was quite lavish with the use of it, but that was before I had any image or audio assets to deal with.

Compared to the memory usage of a handful of image assets, my lavish use of memory elsewhere wasn’t even a problem.

Time came to tighten everything up in Argus, though, because SNAFU isn’t going to get any smaller. I’ve already tackled that in a number of ways, but made my largest gains this week.

In total, I reduced ongoing memory usage by about 84%! That’s huge!

I split the compiled story-file blob up into two blobs. One for the story, and one for the assets.

The assets blob can be trivially mmap()‘ed rather than having to be held in memory. The story file can’t be practically mmap()’ed, but it isn’t very large by comparison. At present it’s about 2MB compared to over 100MB of other assets.

Memory-mapping a file is slightly more arcane under Windows than Posix, but the process is pretty straigthforward overall:

#ifdef _WIN32
 _file = CreateFile(blobname.c_str(), GENERIC_READ, FILE_SHARE_READ, 0, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
 if (_file == INVALID_HANDLE_VALUE)
  return false;
 unsigned int len = GetFileSize(_file, 0);
 _mapping = CreateFileMapping(_file, 0, PAGE_READONLY, 0, 0, 0);
 if (_mapping == 0)
  {
  CloseHandle(_file);
  return false;
  }
 _data= (char*)MapViewOfFile(_mapping, FILE_MAP_READ, 0, 0, 0);
 _size = len;
#else
 size_t len = size_of_file(blobname);
 _file= open(narrow_string(blobname).c_str(), O_RDONLY, 0);
 if (_file == -1)
  return false;
 _data = (char*)mmap(NULL, len, PROT_READ, MAP_PRIVATE, _file, 0);
 if (_data == MAP_FAILED)
  return false;
 _size = len;
#endif

Windows basically boils down to three steps, instead of two, but it works just as well.

The companion to this change is “Don’t clutter up the user’s texture-memory”, because image assets kept getting handed off to GPU memory and never coming back out during the lifetime of the application.

That’s probably not so good. It’s a limited resource, and even though modern hardware has significantly expanded that resource, we shouldn’t just thoughtlessly consume it.

Textures are tracked by usage, now, and textures that aren’t used in several minutes will be quietly purged from texture-memory. If they’re needed again, the system can spend a millisecond or twenty reprocessing them.

That was a good deal more work, requiring the way I handle textures to be reworked completely, end-to-end. The results are pretty satisfying, though.


And surprisingly, that’s all I have this week. Not nearly as wordy as usual. Instead, I’ve got to figure out the knotty bundle of complications in the next bit of narrative. Probably that will be the focus of next week’s progress report.