Kurt McKee

lessons learned in production

Design for "and" and "or"

Posted 12 June 2020 in development and python

Death, taxes, and changing requirements. These are certainties. How you design your code affects how easily you can integrate new requirements. I've made plenty of mistakes in this regard, and I think it's because I made assumptions that didn't hold true.

I'd like to share some of my experiences with you, and the lessons I've learned.

Firmware updates

I work in a global supply chain. We ship standardized stuff to expectant customers. One way that we standardize something is by updating its firmware.

When I first started, the firmware update requirements were simple: there was a single type of firmware that had to be updated, and the requirement was to update all outgoing systems to the latest version.

Easy! I designed the automation to find the latest available firmware and install it.

#
# One-shot code is frequently wrong
# in my line of work.
#

filename = find_latest_firmware_file()
update_firmware(filename)

Then the vendor announced a new requirement: the firmware we had been installing was actually two separate types of firmware packaged in a single file but they had to start releasing the two types separately.

Easy! I rewrote the firmware update code using a loop.

#
# Loops are frequently less wrong
# in my line of work.
#

for filename in find_latest_firmware_files():
    update_firmware(filename)

Things stopped being easy as the number of vendors, product lines, SKU's, and firmware interdependencies ballooned over the years. However, one thing has remained constant: whenever I assume that there will only be one of a thing, I generally have to rewrite that section of the code a year later. That is worst time to do a rewrite.

The hidden assumptions I made eight years ago was that the requirements I was given were immutable, and I designed my code around those hidden assumptions. Since then I've found that loops are extremely helpful for initial implementations; a step in the process may fail or a requirement may change, at which point I usually end up needing a loop anyway.

Lesson: Design for "and"

I use a lot of existing Python libraries to accomplish things at work. Some of them have obviously and deliberately designed for "and".

Look at the doit tool. When defining a task, you have to return a dictionary with a key named actions.

def task_hello():
    return {"actions": [say_hello]}

From the doit documentation:

actions is always a list that can have any number of elements.

There is no singular action key alternative. doit doesn't allow you to skip using a list and then auto-convert your input into a list with one element. If you want a task to perform exactly one action, you'll use the actions key (plural) with a one-item list.

My code has lots of places where I assumed that only one input would ever be needed, like a single firmware type. In my experience, vendors always eventually introduce a second, third, and fourth firmware type that must be updated simultaneously. I've learned that if I design for "and" then my life gets a lot easier when the inputs inevitably expand beyond just one.

Note that one of the implications of this is that you may start designing your code in terms of loops, even when you know that your inputs are currently all single-item lists. I'm not advocating that you use lists and loops. I'm advocating that you consciously consider how hard it would be to rewrite and requalify your code if a second thing has to be done.

Bender, from the show Futurama, angrily asking "You want me to do two things?"

FRU information

In the early days of my career I made a hidden assumption that part numbers never change, and that was reflected in my code by equality expressions:

#
# Singular matching is frequently wrong
# in my line of work.
#

if part_number == expected_part_number:
    take_action()

It didn't take learn to discover that part numbers change all the time, and many times with absolutely no functional difference. (For example, maybe a product is updated due to regulatory requirements like RoHS 2, or perhaps the vendor includes different power cables based on the region it's destined for.)

As a result, I usually assume that there may be additional part numbers in the future, even if I only know of one right now, and I use membership expressions as a result:

#
# Plural matching is frequently less wrong
# in my line of work.
#

if part_number in expected_part_numbers:
    take_action()

This is a subtle but powerful change.

Lesson: Design for "or"

In practice, standardizing on sets has made it easy to group part numbers based on certain criteria, and with set unions I can quickly compare a value against combinations of sets. For example:

#
# Set unions are very powerful.
#

crt_monitors = {"CRT-1", "CRT-2"}
lcd_monitors = {"LCD-1", "LCD-2"}
projectors = {"PROJECTOR-1"}

display = get_output_display()

if display in lcd_monitors:
    enable_screen_dimming()

if display in crt_monitors | projectors:
    enable_auto_power_off()

This design helps prevent hard-coded conditionals that would have to be updated when a new "PROJECTOR-2" part number is introduced. I can add it to the projectors set without modifying any of the conditionals.

Conclusion

Look, at the end of the day, pragmatism has to win over perfection. Whatever code you write has to work and it has to meet the requirements, but it will be very nice if it's maintainable!

So reflect on your own experiences, think about what had to change when new requirements cropped up, and look for ways to design for "and" and "or".