I’ve wound up taking several detours before moving on to stripping strings out of the binary. It’s a good thing I did, too, because I’ve wound up learning a few things.
I started out learning about capability-based operating systems a few weeks ago and how they can be much more secure than traditional operating systems if they’re built that way from the ground up. It’s still early enough in the life of NanoOs that making that adjustment was possible, so I started looking into it.
It took me a while to wrap my head around what a “capability” actually is. The main reason for this is that what it is is very dependent upon the architecture of the operating system. The basic idea is that they’re a piece of metadata that describes something that a process IS allowed to do. Processes are forbidden from doing anything they’re not explicitly granted the ability to do. i.e. A process that is granted no capabilities is effectively powerless. So, a capability is some kind of token that says that a process is allowed to do an operation. But, what is an “operation”? And, how do you define the capability token in a way that’s generic enough to describe any operation?
This is why the definition is dependent upon the OS architecture. In fact, in some OSes, it can’t be defined at all. For intance, I think it would be impossible to turn Linux into a capabilities-based OS. The monolithic nature of the kernel doesn’t provide a clean boundary at which to do a capability check. Fortunately for me, boundaries in NanoOs are very distinct. There are only two: The boundary between one process and another and the boundary between software and hardware. Both of these yield places to do a capability check before continuing with an operation.
Or, to put it in the terms of where I started with this: They both could with a little work. The boundary between processes was pretty straightforward. Processes can only comunicate with each other by sending messages and all message sends ultimately boil down to a single function between non-scheduler processes. So, that was the logical place to put a check. The software-hardware boundary wasn’t as clear cut. The layout of the HAL was structures of function pointers. With the way it was, I’d have to put a check at the beginning of every single function today and every new function going forward. This was obviously unacceptable.
The real desire here was a layer between the function-pointer level and the implementation logic: All function pointer implementations call a single dispatch function that first does a capability check and then routes to the appropriate implementation if the check succeeds. This was a pretty enormous set of changes but they were well-bounded and a good candidate for AI gruntwork. I put Claude on this task and it had the first pass done in about 35 minutes. It then took about another 45 minutes of cleanup and then it was in pretty good shape.
I did have to revise my initial model of what constituted a hardware capability, though. I had initally thought that a capability was a permission to use a specific subsystem function but I discovered that wasn’t specific enough. The reason was timers. Every process has to make sure that the preemption timer is canceled before beginning its context switch because interrupting in the middle of a context switch can cause real damage. (I played that game a long time ago. It’s not fun.) But, I absolutely did not want all processes to have the ability to cancel all timers - I wanted to limit them to ONLY being allowed to cancel the preemption timer. So, I had to extend my hardware capability structure to include a device ID mask. The dispatch function checks permissions on the subsystem and function and individual functions check permissions on the device ID if needed.
Even though the process boundary was cleaner, it was quite a bit more difficult to implement and debug. The reality is that most things in the OS already happen by message passing, the big one being writing output. The console process manages all the console output devices and has direct access to the hardware by virtue of being a kernel-level process. So, the real trick is making sure that other process have the IPC-capability to send output-bearing messages to the console process. I screwed this up several times. And, when I screwed it up, I couldn’t tell what anything was doing. I eventually figured out to give all processes console permissions when a debug define was present so that I could see what was happening until I got everything right.
Finally, after a weekend of work, I successfully transitioned NanoOs to a capabilities-based operating system!! Hooray!!!
This work weighed heavily on my mind, though. Thinking in these terms is really thinking in terms of “real” operating systems, not embedded operating systems that are really meant for 8-bit chips. It’s fine for an embedded OS to have this kind of feature so long as it still serves the purpose of being an embedded OS. The problem was that, at this point, NanoOs really didn’t serve the purpose of an embedded OS anymore.
When I started this work, I was working on an Arduino Nano Every. That system has an 8-bit AVR processor with 48 KB of flash and 6 KB of RAM. I originally had to move off of that architecture not because of space problems but because it was a Harvard architecture which prevented me from using overlays. I always had the goal of being able to run arbitrary programs from the filesystem and that just wasn’t possible on that architecture. Still, I did succeed at having 9 processes running in parallel and a rudimentary shell running from within the main kernel image.
NanoOs has since grown much beyond that baseline. As mentioned in my last post, the filesystem logic alone is 6 KB on the Cortex M0 and probably a little larger than that on the AVR. That’s not a huge concern with 256 KB of flash, but it most definitely is with only 48 KB. Adding in all of the overlay userspace functionality added a ton of code as well.
I wanted to get back to the point of being able to run NanoOs as a very basic embedded OS. I knew that that meant (a) making a whole lot of functionality compile-time selectable and (b) reviving the built-in shell that I had abandoned when I switched to overlays (which was still in a branch that I had archived). The obvious way to go about doing this was by making the functionality’s presence a function of the HAL.
initRootStorage was already a function pointer in the top-level of the HAL. With the way the linker prunes out unused code, just setting that to NULL and putting in a check NULL before calling it would save all of the filesystem space. I realized there were a few other top-level functions like this that I could optionally set to NULL to cut out functionality. So, I made a new HalPlatform subsystem structure to hold these things. The first new member function pointer I added was callFileOverlay. On systems that use overlays, I set this to callOverlayFunctionFromFile. On systems that just use built-in commands, I set this to NULL. That cuts a chunk of the overlay code.
There were a few places, though, that I needed to replace the function called, not simply eliminate it. One was what to do about restarting a shell and the other was what to do about starting a new command process. For these things, I created the restartShell member function pointer and the execCommand member function pointer. On systems that use overlays, I set these to restartOverlayShell and execOverlayCommand, respectively.
That left what to replace these last two with when overlays weren’t in use. For this, I resurrected my original shell and command library from the beginning of my work. I had to update it with modern call equivalents since some of the mechanics of what it did no longer apply. Once I got over that hump, I had two new restartBuiltinShell and execBuiltinCommand functions. On systems that don’t use overlays, I set restartShell and execCommand to these. This eliminated the last of the overlay code.
Unfortunately, after all these changes, I was left with a final binary that was about 60 KB in size. That means it won’t fit on the Arduino Nano Every (which only has 48 KB of flash). I considered trying to use an Arduino Mega 2560 instead and, in fact, created a HAL for it. However, I discovered that flash is not memory mapped on that architecture, which means that all of the read-only data has to be copied into RAM on that system. The amount of dynamic and constant data in the OS now exceeds the Mega 2560’s 8 KB of RAM, so that architecture is out as well. So, it is physically impossible for me to run NanoOs on any AVR architecture that’s available to me at the moment.
I don’t consider this a total loss, though. I’ve now successfully restored NanoOs’s ability to run as a purely-embedded OS on the right architecture and I’ve successfully restored the Commands library to full functionality integrated with the current codebase instead of being archived out in a branch. That’s a win, I think. With the way things are going, NanoOs is now setup to scale from very small systems to (potentially) very large systems if I play my cards right going forward. The fact that how processes are loaded is now a HAL-dependent thing gives some room in the future for me to have even more advanced constructs beyond overlays. I guess we’ll see how well I do.
Onward and upward! HUZZAH!!!
To be continued…