Kurt McKee

lessons learned in production

Hey there! This article was written in 2010.

It might not have aged well for any number of reasons, so keep that in mind when reading (or clicking outgoing links!).

On being held hostage

Posted 1 April 2010 in work

One of the clients I work with had an interesting day recently. You see, they've been using a Linux-based (presumably Unix before that) medical software package for two decades (or so I've been told). Their server now contains almost two decades of patient records, although I'm absolutely certain it's not medical records.

They've recently switched vendors, but have been worried that the original vendor would sabotage the system if ever they caught wind that their software was being abandoned. The new vendor has imposed networking requirements that are incompatible with the old network, and so my company created a bizarre double network, rather than being allowed to work with the original vendor to make sure the old system continued working under the new requirements.

So on the day in question, I found out that I had to switch the network back the way it used to be, because the original vendor's software contains a timebomb: if a new license file isn't uploaded every three months, the server kicks everyone off and refuses to continue functioning. They had hit that limit, and were unable to process claims. I showed up, pulled the ol' switcharoo, and took down their new network during business hours so a tech from the original vendor could remote in and upload a new license file.

When I was allowed to talk with him to find out what holes I needed to poke in the firewall, I was introduced as some guy who sometimes helps with computer stuff, so I had to pretend I was some moron, like a college kid whose mom had called for help. Good thing I can drawl.

Reconfubalating their network wasn't supposed to be an issue, since the new EMR server doesn't require an external internet connection...but then the server crashed anyway. And then the tech accidentally erased every username and password on the old system. Which left the site with no functional software at all.

Le sigh.

The rest of the story short, I got the EMR server to a functional state, and the moment I was told the tech had restored the usernames and passwords I immediately cut off the original vendor's remote access and reconfigured the network so the new system was fully operational again.

...oh, and the new license expires in June.