004 - hard things / templating / booleans and state
Hard things are good
If my concept for eno is actually good, then it begs the question, why has no one done this already? This is a question I meditate on a lot. There are two possible answers that I really hope are incorrect: (1) the concept for eno is a bad idea and (2) the concept for eno is a good idea but it is not practically viable. For this reason, I am heartened when I start work on a new part of eno and discover it is far more complicated and difficult than I expected. It gives credibility to a third potential answer: the concept for eno is good, but the challenge of implementing it is far greater than anyone has been willing to endure. This I can live with!
Templating
A lot of the words people write in documents are recycled from somewhere else. Job seekers send the same cover letter over and over again to prospective employers. Lawyers always reach for a precedent when drafting a new contract. Researchers writing grant proposals stitch together snippets of text from past submissions. Sales people pull text out of marketing materials to put together emails to potential customers. Even when I write this blog every week, I take pieces of it and reformat them to fit into a LinkedIn post.
Software's answer to this is templating. Templating allows you to take a template document, and render it out many times with variable data. Word calls this feature mail merge and it is highly underwhelming. There are other products on the market that offer more sophisticated functionality, allowing you to express fairly complex logic for how your templates should work.
The biggest problem with templating is that it is inflexible. We usually want our documents to be a little bit different from each other. Take, for example, our job seeker. He wants to customize each cover letter to show a unique interest in the particular job being applied for. So, he uses mail merge to create first drafts of the letters which all contain the same paragraph about his fantastic experience. He then edits them one by one, sprinkling a touch of unique flavour into each.
With this painstaking work completed, our job seeker is now ready to submit his applications. As he goes to send the first letter off he decides to give it an extra careful read to make sure there are no mistakes. Imagine his dismay when he finds a glaring typo right in the middle of that paragraph about all his fantastic experiences! Imagine his anguish as he realizes every single letter contains this exact same mistake. While he could fix the mistake in the template and recreate a new set of first drafts, all his meticulous customizations would be lost. Imagine his agony when he concludes his only choice is to go through each letter one by one and fix his mistake. Poor, poor job seeker. If only he had a more powerful word processor…
Booleans and State
You can store two values: 0 and 1. If you put a series of two bits together, you can store four values: 00, 01, 10, and 11. If you put three bits together, it becomes eight values. You can confirm this by writing them all out. Or you can calculate it by raising the number of unique values allowed per digit to the power of the number of digits in your series: 2³ = 8.
One lesson I've learned a few times in software development is that this little fact of math can sneak complexity into your data structures without you realizing it. In my early prototypes of eno I created a data structure called Fragment to store pieces of text in a paragraph:
const Fragment = struct {
text: []const u8,
}
A few days later I added the ability to delete Fragments which I represented as a boolean (true or false).
const Fragment = struct {
text: []const u8,
is_deleted: bool,
}
A few more days later I added the ability to save your work and thereby persist Fragments to disk.
const Fragment = struct {
text: []const u8,
is_deleted: bool,
persisted: bool,
}
Can you spot our math from earlier? Without realizing it, over the course of adding these two attributes I had made it possible for Fragment to be in 2² states. Now, every piece of code that deals with a Fragment has to consider how it should work in each of these 4 states. Or, more likely, these pieces of code don't, and there are potential bugs lurking when a Fragment shows up in a state that was not anticipated in that code path. Emergent complexity like this breeds bugs.
What I like to do in these situations is combine the properties into a single enumerated list of states:
const Fragment = struct {
text: []const u8,
state: enum{ persisted_insert, persisted_delete, staged_insert, staged_delete }
}
Notice that by doing this I had to actually name the negative states. Something that is not a delete is an insert. Something that is not persisted is staged. I am forced to confront exactly how complex my datatype is. What's more, I often find that when I do this in more complex situations, I discover certain states actually shouldn't be possible. For instance, maybe I don't care about persisted_deletes. Instead of having that state I can just destroy the Fragment altogether. That's 25% less complexity to deal with!
In Zig, this pattern is especially nice because of the exhaustive switch statement feature. Consider a code path where we need to do something unique with all "insert" type fragments. We might write that like this:
switch(my_fragment.state) {
.persisted_insert => {
// do something
},
.staged_insert => {
// do something else
},
}
In Zig, this will result in a compile error. The compiler will not allow you to have a switch statement where you do not deal with every single potential case.
switch(my_fragment.state) {
.persisted_insert => {
// do something
},
.staged_insert => {
// do something else
},
.persisted_delete, .staged_delete => {}, // do nothing
}
What I love the most about this is that if one year from now I add a fifth state, this piece of code will throw a build error, reminding me I need to come back and deal with that fifth case.
We also have an RSS feed