November 8, 2016

Web development: Getting 99% correct is relatively easy. (Developer Edition)


TL;DR (Summary) Even though a website seems to be running fine, there might be trouble brewing underneath. If you're a developer who appreciates quality, how does your code & setup differ from mediocre / horrible code? I think it's about moving from 99% (or even 90%) correct to being 99.99% correct and it's about optimizing the correct parts of the system. Though sometimes you also need to ask yourself: What do you want to achieve with these optimizations and is it worth it?

(A list of tools for assessment, development & quality management is attached at the end. This is the »Developer Edition«; if I find the time, I might write the »CEO / Biz Dev Edition« of this article.)


In the last months, I've gotten involved in (or been asked to join) numerous projects where the task was to either improve a technical system (mostly web app) or do a quality assessment. Many of the ones asking me are small companies or startups. And a few times, I've seen that the technical development is partially or fully out-sourced, even when the whole business model relies on the web app. And sometimes, the head of the companies have little knowledge of what goes under the hood. They click on the site and it works. »It's good enough!«

Yes, a CEO or head of business can't know everything. But if the whole revenue stream is based on the web app, it's important that they have at least a basic understanding what's going on, beyond »I clicked and it works«.

And because of this: I've seen some bad code. And I've seen some horrendous code. Same goes for server setups. And deployment processes.

It works. So what's the problem?

How do I explain, that - even though currently the website seems to be working fine - it might not be a great setup. But also for me as a developer, I had to ask myself: If the website works, how would my code and setup be different? What is my added value? What would an improvement achieve? And is it even necessary?

And then I thought about something Elon Musk wrote as a correction, when someone claimed to have developed a self-driving car on his own and he wants to challenge Tesla and their whole engineering:

»This is the true problem of autonomy: getting a machine learning system to be 99% correct is relatively easy, but getting it to be 99.9999% correct, which is where it ultimately needs to be, is vastly more difficult.«

Yes, I develop web apps. Math-wise and stuff, machine learning is much (MUCH) more difficult. But when you create a web site, I think the gist of if can be applied as well.

»Getting a web app to be 99% correct is relatively easy, but getting it to be 99.99% correct is much more difficult.«

(Because when you look at it, it's not about getting from 99% to 99.99% correct, but reducing the error rate from 1% to 0.01%.)

And if you're a »99.99% web developer« and they're a »99% (or even 90%) CEO«, maybe their problem is not yours to solve (or you should turn them into a »99.999% CEO« :-) ).

Questions that need to be asked

So the question shouldn't only be

but should also include

These are the things that I think about constantly. Yes, I can't cover all areas (it depends on the project, how involved I am - e.g. if I am 1 of 5 web developers or if I am responsible for the whole system) but these questions are on my radar.

Figuring out what and when to optimize

I'm not propagating that you need to scale for 100.000 visits / day, not at all. I actually try to assess the requirements and based on that try to keep it as simple as possible and also try to keep the number of »moving parts« (e.g. servers, CDNs, databases, caching layers) and the overhead as small as possible (while keeping options open to scale in the future). Keeping the overhead small lets you focus on the non-overhead work - i.e. the »main« work.

And: There will always be errors every now and then, but the question is how much can you tolerate? Is a 1-2% error rate fine, or do you need errors to be below 0.1% or even 0.01%? And which parts are allowed to have errors? And what should happen when it comes up?

Some examples:

Sometimes »good enough« is good enough

I know developers often have this need to create things from scratch. They can do it much better than previously. And for small systems this might be fine and profitable, but for bigger and for legacy systems the question is: What do you want to achieve and is it worth it?

E.g. I had relaunched small web shops, starting from scratch, and that gained a 20%-40% increase in conversion rate and at the same time also reduced the administrative workload. It paid off after a few months already.

But at the same time, I have created my own web platform that has grown quite a bit in its codebase. The website makes decent revenue, but upgrading it will be quite a pain - I re-wrote everything (away from flat PHP) when I learnt Laravel about 3 years ago - so the code will have different quality levels (from »wow, my first framework«-level to where I am now), depending on how experienced I was when I wrote it and also what the current Laravel version was at the time. The same functionality (e.g. form posting and validation) might be implemented in 3-4 ways throughout the code.

So when taking on a new project (either my own or for a client), I have to ask myself: Is the status quo good enough? Should I update legacy code and invest significant time? Do I need a complete re-write and create a new version (e.g. relaunch)? Do I just need bug fixes and just leave as is and try to make it run as long as possible? Or perhaps - like in the case of Basecamp - have different versions alongside each other and just have long-term legacy support? Or maybe there's another way that wasn't thought up before?

Well, the answer depends on many variables. On what the client wants / needs, what they want to achieve, what the budget is, how technically feasible something is, etc. And this is where experience - i.e. past failures and successes - come in handy.

Tools for web developers

In closing, if you're a web developer, here are some tools that might be helpful in your assessment, development & quality management.

Most of the tools are commercial tools. Many have open source and free alternatives, it's a question whether you want to spend money or rather invest time in setting up your own tools. That's up to you!

Shameless plug for my own tool: After I wrote this article, I wanted to create my own tool that would analyze the »web site quality« based from external information (i.e. without going into non-accessible source code or so), so I spent a few days to create a prototype with SteelCave.io

Frontend performance

Backend Performance and Load Testing

Uptime Monitoring

Quality Assessment

SEO / Online Marketing Monitoring

In essence

TL;DR (Summary) Even though a website seems to be running fine, there might be trouble brewing underneath. If you're a developer who appreciates quality, how does your code & setup differ from mediocre / horrible code? I think it's about moving from 99% (or even 90%) correct to being 99.99% correct and it's about optimizing the correct parts of the system. Though sometimes you also need to ask yourself: What do you want to achieve with these optimizations and is it worth it?

Thanks to Dorthe and Stefan for feedbacking the draft versions of the article.

  • LinkedIn
  • Tumblr
  • Reddit
  • Google+
  • Pinterest
  • Pocket
Comments powered by Disqus