Advent of Code in Production, Days 2-4: Early Development

This continues a series of posts where I explore how to productionize Advent of Code solutions. To summarize what's been done so far: after Day 1, we had a small monolithic application and a simple database. Santa's elves continue to ask for more tools and tech to support their star fruit gathering expeditions. I'm attempting to design a system that will provide those solutions. Some of the requirements for that system are included in the requests themselves, others I'm deducing from context. In any case, I aim to keep the scope of the system focused on the actual solutions we've been asked for, although I do want it to be easily extensible to obvious related problems.

erDiagram
    Backpack||--||Database: "depends on"
    Backpack {
        returns Function "inputs"
        none Parse "list"
        sum Process "parsed list"
    }

Camp Activities

On Day 2, we're not actually asked for a solution to anything. Instead, we're invited to participate in some common expedition activities. This is honestly kind of great. Building tools and solutions is best done with a solid understanding of why they're needed and how they'll be used. What we learn is that the elves spend what seems like a lot of time competing with each other for preferential camp spots (thousands of rounds of rock, paper scissors!). And then shortly, that the camp setup did a pretty poor job of accounting for future needs and has to be rearranged the very next day.

Rock, Paper, Scissors

If we read the rock, paper, scissors challenge as a request for something, then I'm inclined to see this as an attempt to solve a social problem with technology. That's not likely to ever work. But even so, we may be better off building it anyway. Solving social problems requires a lot of trust. The elves are pretty happy with us, and we're better off if we keep that going. It's extremely common to be asked to build this kind of tech solution for social problems. We should take this as an opportunity to talk about their wants and needs. We should probably try to discourage them from pursing tech solutionism in favor of more planning and open communication. But, if building what they ask for also helps to build our relationship with them, then I'm confident it's worth it in this case. It's an easy enough tool to build, and we can guide them in the direction of simpler solutions. Something like a rock, paper, scissors score keeping tool, rather than a game solver. We can plug that into the database if they want historical results, but we'll wait to see if that's something they even want.

Rearranging

On days 3 and 4, we're back to work, and discover new problems the elves need tools to help solve. First is that a lot of the backpacks have been packed improperly and they need to be rearranged. Second, the camp is set up incorrectly and also needs to be rearranged.

Backpacks

To help with repacking the backpacks, we're given packing lists for all of them, and we're asked to find the lists where items are duplicated across multiple compartments. Presumably these are the same backpacks from day 1, which means there's a little extra wrinkle for us: our existing data model. We previously modeled the contents of the backpack as being simply food. That's not obviously true anymore. There are clearly many types of supplies. So, in addition to solving this problem, we're also going to have to evolve our data model.

Now, we haven't been asked to build an inventory system. And I am, in fact, very eager not to build an inventory system. So that's not what we're going to do. The elves seem to already have a working solution for that. What they need help with is downstream from there; things like packing and logistics. So, I'm not going to track supplies as their own entity in the model. Instead, we'll track the contents of storage systems like backpacks and the camp inventory. So, our Backpacks go from containing many Foods to containing many Supplies. There are many types of supplies. We care about some of their properties, like calories. But there's likely other relevant properties, too. We're already tracking calories as a property. We can redefine that as a generic value and record a separate unit of that value. One such type would be calories, but this way we can have others, and provide some sensible handling for them in our user interfaces. And as long as we're expanding our Supply data model, let's include a quantity. So far Backpacks only seem to have one of any given item, but that could easily change. And other storage options are very likely to have more than one. This should allow us to do things like prepare better packing lists, help elves find who's carrying an item out in the field, and find where items are stored. And since we're considering other storage containers, we should probably also genericize our Backpack model, to be a Container with a type.

erDiagram
    Expedition ||--o{ Container : includes
    Container ||--o{ Supplies : contains
    Supplies {
        Number value
        Text unit "Calories|Range|Doses|etc"
        Text item_code
        Number quantity
        Number priority
    }
    Container {
        Text type "Backpack|Crate|etc"
    }

One note about this schema migration: renaming columns and tables can be pretty fraught. We could probably get away with it in this case, but as systems get more mature and store more data it will rapidly approach impossibility. The more viable solution is to do just about anything else. We could document the historical context. We could create views with aliased names for normal use. We could handle and clarify the discrepancy in our code's data access layers. We could potentially start over with a new table and migrate our data. I'm not worried about which solution we use for this exercise, any of them will work. I just thought it was important to call that out.

The actual immediate request with this bit of tooling is to find backpacks that contain duplicate items. We'll be given packing lists from those backpacks to analyze. So we'll repeat our now familiar parse and process interface design. But there are a number of very obvious extensions to this basic processing that are now very easy to do. We could get Backpacks that contain some item, or don't contain it. Or do some statistical analysis about the distribution of items or item priorities. That's just off the top of my head. We won't get ahead of ourselves by building them just now, but we can start getting the elves used to the idea that there's more we can do for them and start collaborating on solutions to problems rather than simply fielding a list of requests.

Camp

They've broken up the work of rearranging the camp to make room to unload supplies by assigning elves to work in sections. The camp is divided into 99 numbered sections, and each elf will work in a continuous set of those sections. It's not clear what—if any—logic is applied to making these assignments, and they are really inefficient. The elves want some tools to help find the overlapping work assignments. They've paired up to compare assignments and want to know which pairs are fully redundant. That's not the worst idea, actually. It should enable a quick high order analysis, and if you're doing these things by hand that might be all you can make time for. It seems they've really been needing help, though.

Once again, let's consider the data model. I think we have Sections and Assignments. They are given to us paired together, and that's information we need to retain for our immediate purposes. I'm skeptical it's a good way to model the data for a longer-term purpose. But I can take this cue from how they're already working. Assignments can be grouped together, so let's say we also have Assignment Groups. It's not clear if Sections have any information about them other than their ID number, but I'll err on the side of extensibility and make that a table in our database. It seems likely that they do have other features we'll need to know about, and that they might be meaningful in other contexts than just work assignments. If I was to guess, I'd say those sections are probably a consistent addressing scheme for the lifetime of the camp. Assignments are also a table, and we'll want an Assigned_Sections table to maintain the relationship between the two. Assignments have some temporality to them. Which is to say they begin and end. There's also some context to them. In this case, the elves are being assigned to cleanup. But it's again likely that other kinds of tasks exist. For now, I'll avoid making that an enum, and just let it be free text. The elves seem to be pretty good at producing consistent data, so I'll favor flexibility in this case. And finally, we want Assignment Groups so we can do our initial pairwise analysis.

erDiagram
    Assigned_Sections ||--|{ Section : "has many"
    Assigned_Sections ||--|| Assignment : "has"
    Grouped_Assignments ||--|{ Assignment : "includes many"
    Grouped_Assignments ||--|| Assignment_Group : "is"
    Assignment {
        Date date
        Text description "Clean|Repair|etc"
    }
    Section {
        Number Id_Number
    }
    Assigned_Sections {
        ID Assignment fk
        ID Section fk
    }

With our updated data model establish, the feature we need to build is pretty straight forward, at least in terms of the system model. It will parse and process a list of sections in an assignment group. The sequence diagram is basically identical to the other features we already built. But it's again easy to expand to more analyses. And if we can include a location for our Containers, even an optional location, then we're really talking. We can incorporate that information into the way we organize assignments, and start making improvements, not just chasing whatever pain point is most immediately relevant. Here's that whole entity model.

erDiagram
    Assigned_Sections }|--|{ Section : "has many"
    Assigned_Sections }|--|| Assignment : "has"
    Grouped_Assignments |o--|{ Assignment : "includes many"
    Grouped_Assignments ||--|| Assignment_Group : "is"
    Assignment {
        Date date
        Text description "Clean|Repair|etc"
    }
    Section {
        Number Id_Number
    }
    Expedition ||--o{ Container : includes
    Expedition ||--o{ Assignment : involves
    Container ||--o{ Supplies : contains
    Container |o--|| Section: "in a"
    Supplies {
        Number value
        Text unit "Calories|Range|Doses|etc"
        Text item_code
        Number quantity
        Number priority
    }
    Container {
        Text type "Backpack|Crate|etc"
    }

Parting Thoughts

I think it's interesting that the elves themselves are never included in the data they give us. No names, or employee numbers, or even something like job titles. Of course, Advent of Code is just a hobbyist community event, which I'm bending all out of shape to do this take on it, so that really doesn't mean anything. Except, this only works at all because the event itself is presented narratively. The elves are characters in the story, but not in the data. I wonder why they would leave themselves out. Is it an aversion to being tracked? Or fear of being blamed? I could certainly sympathize with that. There are some pretty obvious dysfunctions in their working environment. It's easy to believe a lot of their processes developed as workarounds to that reality. In the real world I would want to try to improve that situation along with everything else.

And, as before, if you'd like to peruse my mostly unrelated, one-off golang solutions to these challenges, be my guest. I am even more behind with those than I am here, but don't let that stop you.

Cover photo by Meruyert Gonullu