I’ve had the ability to kill rogue processes since enabling preemptive multitasking. The problem was that it took a second shell to do it. This was possible on my Arduinos - when I had the hardware UART connected. It was not possible on my simulator and it stopped being possible on the Arduinos once I started using my UART hardware to talk to my eZ80. (More about that some other time.) This was super annoying. The obvious answer was to implement a way for Ctrl-C to work properly, which meant implementing POSIX signals.
I’ve had the basic idea for how to do this floating around in my head for quite a while. What I did was to add a callback that’s called from within the process’s running context right after it’s resumed. It was a little complicated, though. For one thing, I realized that - if the callback was set - it had to be called unconditionally regardless of what was passed in through the resume call. For another, from within the callback, I had to have a way of determining whether what was passed in was intended for the callback or if it was something that was intended for the running process (since the underlying coroutine framework I’m using assumes that values can be passed in directly via the resume call). If what was passed in was a NULL pointer (which is the usual case), no issue since the callback requires a special value and a NULL pointer obviously can’t point to a special value. But, how do you determine if a non-NULL pointer is intended for the callback or the underlying process?
The simple answer is that you need a magic value - or what I wound up referring to as a “signature” - to indicate to the callback that it had received something intended for it. But, it’s a little more complicated than that. You only get one (1) void* argument to the resume function and the callback would obviously need other parameters besides the signature, so a pointer to a structure would need to be passed. But the callback can’t attempt to evaluate a full structure because what’s passed in might not be a structure intended for the callback. So, I had to make sure that the signature was the first value of the structure. When the callback detects a non-NULL argument, it treats it like a pointer to a uint64_t and evaluates it as a signature. Only if the signature matches what’s expected does it go on and treat the rest of the pointer like an argument structure.
So, with that much worked out, I went about implementing the parameters to a sighandler_t and a default signal handler. For now, I’m only interested in killing the process, so I only implemented a handler for SIGINT and SIGTERM, which just tells the scheduler to kill the PID of the running process. I then had to add logic to the console process to detect ^C being entered from a console and directing it to the process associated with the console. Tedious, but pretty simple.
Once I got all the plumbing in place, I tried running my tightLoop program as a foreground process and then hitting Ctrl-C. To my utter astonishment, it worked correctly on the first try!! HOORAY!!! This work, however, turned out to be a bit like opening Pandora’s box…
The entire system is predicated around passing messages among processes to accomplish work. Messages have three basic components: A type, a data pointer, and a payload size. I’ve always used the type as an index into an array of function pointers to determine what the command handler should be. And, of course, I bounds check that to make sure that I don’t accidentally index into a function that doesn’t exist, so that much is fine. On rare occasions, though, I’ve been bitten by sending a message to the wrong process and the type falling within the legal bounds of handlers for that process. When this happens, it’s an absolute mess. It can be difficult to tell that it even happened and very circuitous to track down. One of the things that has made debugging these situations difficult is that I wasn’t able to tell what process the rogue message had originally been intended for.
It dawned on me that the signature validation technique I used for the resume callback could also be used to validate the incoming messages to a process. Naturally, my brain was stuck in the thinking of the work I’d just done, so my first draft of the fix was to do the exact same thing and add a signature field to all of the message structures with a unique, 64-bit value per process. Then, I hit a case of sending a purely asynchronous message and had a dope-slap moment. With a purely asynchronous message, the caller isn’t waiting on a response and therefore doesn’t send a structure-based data payload. It’s just a “fire and forget” message that gets sent without a payload.
The solution to this became immediately obvious. The type field of my messages is a 64-bit integer. I had made it that wide initially using the old-school C philosophy of “don’t constrain the programmer”. What I could do was use a 56-bit signature value and leave the lowest 8 bits as the command handler index. I’ll be very surprised if any of my processes ever have more than 256 commands they support and - if by some strange scenario they do one day - I can just change the signature for that process to use 48 bits instead. So, I then went back and painstakingly converted all the commands I’d just converted to using 56-bit signatures bitwise-ORed with the command enum value. I then went on and finished the rest of the asynchronous messages.
Then, I had another realization. A long time ago, I put in a mechanism to (hopefully) detect when a process’s state object had been corrupted. Process state objects are put at the top of a process’s stack, so the way they become corrupted is by an adjacent process overflowing its stack. It’s bugged me for a long time that I could detect that a process’s state was corrupted before I attempted to run it and thereby avoid catastrophe in the system but I couldn’t detect the corruption (stack overflow) when it happened. I realized that I could put a 64-bit value right before the declaration of a process’s state object and set its value to a special value. I could then set a pointer to this value in the state of the previous process. That would give me the ability to check for a stack overflow by seeing if the value at the pointer was the expected special value.
I implemented that special-value mechanism, too! Now, FINALLY, I can detect a stack overflow right after it happens and kill the offending process. That obviously won’t prevent a neighboring process from becoming corrupted because the damage will have already been done by then, but it will allow me to prevent any further damage by the one that went rogue. It also allows me to clear out the offending process’s slot and reuse it for something else.
So, three new features that make the system more stable and robust. This is starting to look something like a real operating system at this point. It’s a system a user can actually recover from a mistake in. That’s pretty cool in my opinion!! Onward and upward!!!
To be continued…