Development Progress Report, week 60

Well, I’ve built a lot on top of last week’s successes. Splitting the save-files into multiple sub-files was a good move. Following the logical thread of that, the compiled story-file is now a set of sub-files as well; and I was able to do something pretty awesome with that.

Continue reading Development Progress Report, week 60

Development Progress Report, week 59

Last week, there were a bunch of things that weren’t quite right. Attempts to improve font-handling were bungled, music wasn’t restoring properly when the game was reloaded (which I thought I fixed, but I wasn’t entirely correct), and image-transitions were kind of fouling up my cool game-loading effect.

This week, that’s all sorted.

Continue reading Development Progress Report, week 59

Development Progress Report, week 41

There’s been a big dip in the development schedule. Family urgencies and other commitments sapped a lot of my time and energy, but I’m back to developing again.

Time to look at progress for the previous week, and maybe a few other treats, besides.

Continue reading Development Progress Report, week 41

Development Progress Report, weeks 39 and 40

Two weeks since the last update, and things have been going a bit slowly. Headachey and nauseous today, which is certainly not helping.

There have been a number of code-improvements that simplify authoring, especially when casting characters into scenes, and similar meta-programming tasks. That’s all to the good.

The writing has been a bit slower, and I think it is because I got hung up on a particular scene. Continue reading Development Progress Report, weeks 39 and 40

Development Progress Report, weeks 37 and 38

Missed a week, due to life and commitments, so this is a double week, with a blob of promised telemetry source-code.

Story-compiler

The Story-Compiler is a nineteen-pass tangle. Basically, any time I need it to do more, and whatever-it-is doesn’t cleanly fit into any of the existing passes, I just add another pass.

This leads to some interesting precedence-of-operations issues. This pass should happen before these other passes, but… oh, it relies on this other pass to go earlier.

That may need a tidy up. Also, now that I’m expanding and replicating parts of the source on its way through the story-compiler, I’m pretty sure the way I’ve been tracking the start/end positions of scenes in the intermediate data-structures is starting to drift.

Not enough to break anything just yet, but I’m sure it will, as the compiler uses a lot of per-scene data. In short, I need either a better way of tracking which line of the intermediate story-data belongs to which scene, or a better way of inserting/deleting lines of the intermediate story-data.

The engine itself has no such troubles, being that it breaks everything out of the final compiled story into separated scenes and performances. I suppose I could actually tear the intermediate data into parts, and reassemble later, but that would make some tasks harder, rather than easier.

I think, for now, better tagging of intermediate lines is probably the way to go. Probably.

Syntax

I’m slowly replacing some of my original syntactical constructs with newer things. Actually, in the compiled story file, things are still pretty much the same, but I’m using the compiler phase to allow for a more sensible syntax for what were previously some historically awkward constructs.

The idea is to make everything a whole lot simpler. Here’s an example:

Scene 115
Meta
 Summary: Alice and Bob give either Christine or Danielle (whoever it worked out to in scene 100) a lift to the next town.
 Arc:The Mysterious Hitchhiker
 Follows:100
 Precedes:130
Roles
 Copyfrom:100
 // Set up aliases for cast we copied from scene 100
 alias:1:driver
 alias:2:passenger
 alias:3:hitcher
 role:4:name=Craybourne [alice]
 role:5:name=Drayper [bob]
Requirements
 None
Performance
{
passenger,hitcher:{Name:[driver]} put the car in gear, flipped on the indicators, and checked {herhis:[driver]} mirrors. {?role:[driver]:Drayper|He nervously glanced this way and that before finally pulling awkwardly out onto the road|There was only a moment's pause before she pulled the car smoothly onto the highway, and accelerated}.
...
}

Trust me, that’s a whole lot nicer.

Story

Work’s coming along well on the next stage of the story. It requires some parallel tech that I’m throwing together on-the-fly, as I need it. It’s slow, but steady, and there’s another big plot-twist, besides.

The nature of that twist is something I want to revisit before it is set in stone. Does it need a trigger warning? Does it go farther than I’m comfortable with for the overall tone of the story?

Possibly.

This actually gives me two options.

I could either make some adjustments to the scene so that it is a good deal less whatever-it-is (trying not to be spoilery is hard!). Or, I could do that and present it as an optional form of the story. Maybe stick some trigger-warnings on the options screen, and eliminate/vary characters or a couple narrative elements based on what you choose not to see.

The second one sounds more complicated, so I’ll probably end up at least writing the code for it eventually, even if I don’t take advantage of it myself.

For now, I might archive the problematic bit of narrative, and tweak it. I’ve got a softer version that should still fit the bill narratively.

Game Telemetry

At the IGDAM get-together last week (one of the life bits that ironically got in the way of getting the devblog done), I promised to post my telemetry code. Telemetry in your alphas/alpha-demos is important, I think. That’s around the time you want more data about how people actually are playing your game, but as an Indie, you can’t exactly afford any kind of structured playtest – but as long as you have a cheapie web-host that you can drop a script on, you can get game-telemetry, and thus feedback that will help inform your design and polish decisions.

The code here isn’t particularly arcane, and should build under both Visual Studio on Windows and G++ on Linux. Basically, if you know C++, then there’s nothing terribly difficult in this.

Telemetry.h:

#if ENABLE_TELEMETRY
#include <deque>
#include <string>
#include <clocale>
#include <locale>
#include <codecvt>
#include "SDL.h"
#ifdef _WIN32
#include "windows.h"
#else
typedef uint32_t Uint32;
#endif

class TelemetryEvent
{
public:
 typedef enum { none = 0, startup = 1, shutdown, like, dislike, loadstart, loadend, savestart, saveend, characterpick, footnote, loaderror, runtimerror, titlescreen,choice } code_t;
 TelemetryEvent(code_t n) { eventcode = n; timestamp = SDL_GetTicks(); }
 TelemetryEvent(code_t n, const std::wstring &str) { eventcode = n; timestamp = SDL_GetTicks(); data = str; }
 ~TelemetryEvent() = default;
 code_t eventcode = none;
 Uint32 timestamp = 0;
 std::wstring data;

 std::string to_string()
 {
 std::string result;

 result += std::to_string(eventcode);
 result += ",";
 result += std::to_string(timestamp);
 result += ",";
 std::string tmp(data.begin(), data.end());
 result += tmp;
 return result;
 }
};

class AppTelemetry
{
private:
 bool WS_init = false;
 std::deque<TelemetryEvent> events;
 void Transmit();
 bool enabled = true;
public:
 AppTelemetry();
 ~AppTelemetry();
 void SendEvent(const TelemetryEvent& te);
 void Poll();
 void Flush() { if (!enabled) return; while (events.size()) Poll(); }
 void Enable(bool b) { enabled = b; }
};

extern AppTelemetry TelemetryObject;

#endif

Telemetry.cpp:

#if ENABLE_TELEMETRY
#include "Telemetry.h"

#include <iostream>
#include <thread>
#include "EngineIO.h"
#ifdef _WIN32
#include "Winsock.h"
#pragma comment(lib,"ws2_32.lib")
#else
#include <arpa/inet.h>
#include <netdb.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <unistd.h>
#include <netinet/in.h>

typedef uint32_t Uint32;
typedef int SOCKET;
typedef sockaddr_in SOCKADDR_IN;
typedef sockaddr SOCKADDR;
static const SOCKET INVALID_SOCKET=-1;

void closesocket(SOCKET s)
{
 close(s);
}

#endif



AppTelemetry TelemetryObject;

#define TELEMETRY_HOST "www.example.com"
#define TELEMETRY_HEADER "GET /telemetryscript.py HTTP.1.1\nHost: " TELEMETRY_HOST "\nX-Xtelemetry: "
#define TELEMETRY_PORT 80

AppTelemetry::AppTelemetry()
{
#ifdef _WIN32
 WSADATA WsaDat;
 if (WSAStartup(MAKEWORD(2, 2), &WsaDat) != 0)
 { // failed

 }
 else
#endif
 WS_init = true;

}

AppTelemetry::~AppTelemetry()
{
#ifdef _WIN32
 if(WS_init)
 WSACleanup();
#endif
 WS_init = false;
}

void AppTelemetry::SendEvent(const TelemetryEvent& te)
{
 if (!enabled)
 return;
 events.push_back(te);
}

void AppTelemetry::Transmit()
{
 if (!enabled)
 return;
 if (events.size() == 0)
 return; // No work to do
 // Get the first event from the queue
 TelemetryEvent te(events.front());
 events.pop_front(); // And remove it from the queue

 SOCKET Socket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
 if (Socket == INVALID_SOCKET)
 return; // Nuts. Can't create the socket
 struct hostent *host;
 if ((host = gethostbyname(TELEMETRY_HOST)) == NULL)
 {
 closesocket(Socket);
 return; // Can't look up the name.
 }
 SOCKADDR_IN SockAddr;
 SockAddr.sin_port = htons(TELEMETRY_PORT);
 SockAddr.sin_family = AF_INET;
 SockAddr.sin_addr.s_addr = *((unsigned long*)host->h_addr);

 // Attempt to connect to server
#ifdef _WIN32
 // The Windows version of this function returns SOCKET_ERROR on failure.
 if (connect(Socket, (SOCKADDR*)(&SockAddr), sizeof(SockAddr)) == SOCKET_ERROR)
 {
 Output(DEBUGGING, "Winsock connect error "+std::to_string(WSAGetLastError()));
 closesocket(Socket);
 return; // Connection failed
 }
#else
 // Linux returns 0 on success
 if (connect(Socket, (SOCKADDR*)(&SockAddr), sizeof(SockAddr)) != 0)
 {
 closesocket(Socket);
 return; // Connection failed
 }
#endif

 // TODO: Send the data here.
 
 send(Socket, TELEMETRY_HEADER, (int)strlen(TELEMETRY_HEADER), 0);
 auto msg = te.to_string();
 send(Socket, msg.c_str(), (int)msg.length(),0);
 send(Socket, "\n\n", 2,0);

 // Shutdown our socket
 shutdown(Socket, 1);

 // Close our socket entirely
 closesocket(Socket);
}
void AppTelemetry::Poll()
{
 if (events.size())
 Transmit();
}



#endif

telemetryscript.py: (Sorry about the way indenting got eaten up here, but you can figure it out)

#!/usr/bin/python

import os
import codecs
import time

class FileLock:
 def __init__(self, filename):
 self.filename = filename
 self.fd = None
 self.pid = os.getpid()

 def acquire(self):
 try:
 self.fd = os.open(self.filename, os.O_CREAT|os.O_EXCL|os.O_RDWR)
 # Only needed to let readers know who's locked the file
 os.write(self.fd, "%d" % self.pid)
 return 1 # return ints so this can be used in older Pythons
 except OSError:
 self.fd = None
 return 0

 def release(self):
 if not self.fd:
 return 0
 try:
 os.close(self.fd)
 os.remove(self.filename)
 return 1
 except OSError:
 return 0

 def __del__(self):
 self.release()

def storetelemetry(ip,tel):
 f = codecs.open('/home/whatever/data/dtelemetry.txt', 'a','utf-8-sig')
 f.write(ip)
 f.write(':')
 f.write(tel);
 f.write('\n');
 f.close()

print "Content-Type: text/plain\n\n"
remote_ip=os.environ['REMOTE_ADDR']
telemetry=os.environ['HTTP_X_XTELEMETRY']
lock=FileLock('/home/whatever/data/lock000')
while not lock.acquire():
 time.sleep(1)
storetelemetry(remote_ip,telemetry)
lock.release()

I call it like this:

#if ENABLE_TELEMETRY
 TelemetryObject.SendEvent({ TelemetryEvent::characterpick,selected_character }); // send chosen character name.
#endif

Call TelemetryObject.Poll() in your mainloop somewhere, and TelemetryObject.Flush() before you exit, to make sure any telemetry messages have been sent.

The resulting file on your web-host (dtelemetry.txt) should be simple enough to parse and extract useful data from. It has a millisecond timestamp for each event, an event-code and whatever optional extra data gets sent along.

Caveat: The whole thing is single-threaded, so the act of generating an event can stall your game. You want to tweak it to run in a thread? Go for it. Just watch out, because I’m pretty sure that under Windows, your Winsock code needs to be initialised in the same thread that calls on it. The Linux code should be gold.

 

Show me the code: cpuid and popcount

 

So, modern CPUs have built-in functions for a lot of things, these days, which can offer you blisteringly fast access to some otherwise rather awkward and slow algorithms.

In my case, I wanted to access the CPU’s popcount function (count how many set bits there are in a value). The trick is that either the CPU supports it or it doesn’t, and different systems have different ways of testing if it does, and accessing it if it does. Of course if the CPU doesn’t support it, you still have to do it in software anyway.

So, I thought I’d write something extensible, and I thought I’d share.

Continue reading Show me the code: cpuid and popcount

Rewriting text output

Rendering text in a graphical environment is … let’s call it ‘involved’. Assuming you want to output a single line of text in a single typeface and colour, that’s pretty easy. The interface libraries will even probably be willing to word-wrap it for you.

If you want to do other things, like mix plain text with italics or bold or text in other colours, well suddenly you’ve got dozens of hoops to jump through.

Well, I did want that, so I had to work for it.

Continue reading Rewriting text output

Making inferences

Earlier on in the development of Argus, I made the decision to not bother checking if a character in the story already knew a piece of information before they ‘learned’ it.

My thinking at the time was that the act of looking up the information was essentially the same cost in time and resources as learning it. So, we may as well just learn it. If it altered what they knew, fine. If they already knew it, still fine. Nothing changed.

Then the knowledge inferences/assumptions system got more advanced. Story characters learn about each-other as the narrative progresses. Knowledge about the gender and approximate age-group of speakers is acquired by listeners, when the character speaks, and information is combined with other pieces of knowledge to determine, for example, just how characters should refer to each-other during the narrative.

Eventually the process of looking up lots of disparate pieces of knowledge, filling in with inferences and assumptions, and then producing a reference-model for referring to other characters (in a variety of levels of formality) started to get a bit expensive.

So, I finally bit the bullet, and added a method to the knowledge graph to test if something was already known before we tried to get a character to learn it. That way we can assess if knowledge the character consists of any new knowledge that might require updating their assumptions/inferences.

That means double the workload for any learning a character does (which happens a lot), but the reduction in reference-model updates actually pays back that cost almost fifty times over. The story can be processed even faster than before, with fewer spikes in CPU usage.

Overall, that’s a win.

It did lead me to code this peculiar little construct, though:

KnStore* KnStore::find(statement_t &statement, const std::wstring &p)
{
 if (statement.size() == 0 && this == nullptr)
   return (KnStore*)1;
...

DO NOT TRY THIS AT HOME!

Yes, I’m intentionally creating a bad pointer. I already had a find() method that is used to locate knowledge insertion points. Producing a contains() method (does the knowledge already exist) could use exactly the same logic (that is, call find() and test the returned pointer), but always ran off the end of the tree to a null KnStore instance, if and only if it was successful.

That was easy to subvert, and I can return a dummy pointer that no code path ever dereferences (I could also have thrown an exception, but at an additional cost).

Honestly, if I ever saw code like this in someone else’s software, that would immediately raise questions and not a few red flags. Surprisingly, in this case, it’s the perfect solution for a very specific problem.