Mental maps for navigating software systems

The core, fundamental task in software engineering is to build mental models of the systems we work on. Or mental maps, if you will. That's the metaphor I'm using for this article, so I hope you will. Peter Naur described this as theory building. Everything else we do—writing code, writing tests, designs, estimates, architecture, all of it—flows from that. Building these mental maps is challenging work. It depends on expertise and experience. And it takes time to interact with the system and learn how it works.

Naur also noted that it's more or less impossible to document the models we build. At least not in a way that allows other people to recreate the model. We can't write the theory down. We can't draw out the mental map. It does work a little better to talk through it live, in real time. That's because it's a matter of interaction rather than communication, and I'll come back to that.

Maps and territories

We work in complex, dynamic, often chaotic systems. Any model of any system is, necessarily, a simplification. They're abstractions. They're projections into a lower order of complexity. In much the same way that maps are projections of a 3d surface into 2d space. This is helpful for us, because it allows us to better make sense of them. But it's also just necessarily true, from an information theory perspective. In order for a logical model to perfectly capture the complete state of another system, the model would have to be even more complex than the system it's modeling. That means our mental maps are lossy relative to the system we're mapping. To be clear, that's not a bad thing. In fact, it's a very useful thing. That's what makes them useful to us in the first place. But it's important to understand it to make the best use of it.

All models are wrong, but some are useful

You may have heard the idiom that the map is not the territory. The word is not the thing. Our mental maps are imperfect. We make them up from symbols, representations, abstractions, and analogies. In the sense of whether they correctly reflect the systems we're building, they don't. Our models are wrong. But that doesn't stop them from being useful. Consider the difference between a road map and a flood map. Should a map show borders or elevation? What about rainfall or prevailing winds? The answer is that it depends on what you're doing with that map, and any of them could be valid. The same is true of our mental maps.

Mapping the territory

One reason it's hard to build these mental maps is that we can only map the things we've encountered. The map is a metaphor for our knowledge of the system. It's our understanding and intuition about how it's composed and how it behaves. We can only map out the places we've been.

Another reason it's hard is that building these mental maps is not simply a matter of knowledge. Peter Naur's phrasing is very good, because what we actually need is to develop a theory of how it works. Reading about other people's theories only shows us to do that in very limited ways. We really require interacting with the system, to observe how it responds. One thing about "systems" is that there's no objectively correct way to define their boundaries. Even the borders of our mental map are a (often subconscious) decision about where it makes the most sense to draw them. The "systems" we build are computer or software systems, yes. But it's more useful to view them as sociotechnical systems. They're composed of both people and machines, interacting and communicating with each other; dependent on each other.

The reason that interaction, rather than communication, is the limiting factor on building your mental map is because you are part of the system. Your mental map is a reflection of how well connected (in the sense that a graph can be well connected) you are within the system, and how you view and understand those connections. That means you enhance your mental map of the system by becoming more integrated into the system. That's also why documentation is so unhelpful, but discussion can help quite a bit. The people you would have that discussion with are also part of the system, and you can observe how they react in response to your questions, assumptions, or choices. You also become more situated within their mental map, and more connected to them and the rest of the system.

Exploration, then navigation

We build our own mental maps by exploring and discovering the system. And critically, by observing how the system reacts to stimuli. How it behaves in certain conditions. How it responds to change. We can be helped along in that process. Other people can show us points of interest. We can try to retrace someone's past journey through the system. But ultimately, we have to do this for ourselves. Making our map useful to someone else requires a lot of shared context. Our maps don't exactly have GPS (well, robust observability would help). What we have are sort of mental landmarks and loose sketches. Getting those to line up isn't trivial.

First you learn to read, then you read to learn

Building and maintaining these mental maps is the foundational task of software development. Most of the time, it's an assumed task. A sort of dependency or prerequisite that the developer is just expected to satisfy in the course of pursuing some other, more concrete goal. But there is a time that's not the case. A time when making the map is the goal. We call it onboarding. It's the period of building early connections to the system. Doing that initial discovery. It's like learning to read. Eventually it's just assumed that you can read, and then you have to read to learn.

But, as I hope I've made clear, that map making doesn't stop. We're continuously refining our mental map. We expand it, we add detail, we remove unimportant elements, and we make more specialized mental maps for more specific purposes. We need to constantly discover and rediscover aspects of the system and revise the maps accordingly. We can make that easier to do, and help ourselves be more successful doing it. If your mind jumped to documentation, that's not surprising, but it's also not a particularly good option. As software developers, we talk about documentation a lot. We put off writing it. We bemoan when other people haven't written it. But remember, we cannot document our mental maps. The better way forward is to make the system more discoverable. We can do that by clearing obstacles. We can signpost interesting or important things. We can build safe and easy paths toward goals people tend to have. And perhaps most importantly, we can equip people to go off the path with confidence.

Discovery

A system is more discoverable when the links between components are more explicit. For a software system, those links are very often technical dependencies. And very often the way to make the links explicit is with hyperlinks. But things like package managers, application manifests, or shared dotfiles also serve this purpose. The idea is to create paths for people to follow when trying to understand how things work. It makes it easier to answer questions like "where did this come from," or "why is that necessary?"

Clearing paths

The biggest obstacle to exploring a system is just poor discoverability. The next biggest is elaborate rituals that have to be followed to be able to work on it. Like a large collection of tools that have to be uniquely configured for each user, or each device. Or a special system configuration that has to be precisely replicated. Even worse if that configuration has elements no one knows exist until someone discovers something that doesn't work when they try it. Solutions to these problems might be to package configuration along with code, use common tools, and ensure the defaults are usable. Establish and follow common conventions, and call out when you deviate from them. Some level of automation will also help, as long as the automation itself doesn't hamper discovery. The goal is to make it easy to move around in the mental space of the system, without clearing away so much of what's involved that you lose the landmarks.

Leaving the path

In building software, there are things we do repeatedly. We build, test, and deploy changes, for instance. There's a lot of value in making those things easy, safe, and discoverable. But we're also building new things. We make new products and new features. We reach new limits on our scale or experience service disruptions, and we do entirely novel and experimental work to resolve those issues. Essentially, we're venturing into the unknown. We are both drawing the map and creating the territory. It's actually really impressive when you think about it. We can set ourselves and each other up to do this successfully, and safely. To do that, we can provide tools to help with exploration. Create spaces where it's safe to experiment. Design failure domains so that mishaps are contained. Make use of backups and reproducible builds so that bad changes can be undone. And stay in frequent contact with each other, so that help is readily available.

Most of the time, when we talk about these things it's in the context of onboarding. But I think that's short sighted. These are qualities of the system that make it easier and safer to learn and explore. Those things don't stop after a week, or a month, or whatever your regular onboarding timeline is. It happens continuously, forever. For as long as the system is operating, it is also changing. And for as long as the system is changing, we need to learn how it works. If creating those mental maps is the most important thing we do as software developers, then making the system easier to explore literally makes us better developers.

Cover photo by Alex Andrews