Author Archives: Matt Warren

escape artistThere are many software platforms that offer ‘magic’ like ways to accomplish things. Ruby on Rails has a lot magic going on – pass in a string argument and it gets automatically pluralized, converted from snake case to elephant case, inferred as a class name in the global namespace, instantiated and connected to set of URL routes.  Magic code is dangerous because it’s easy to use without needing to understand how it really works, and leads to a false sense of confidence that you know what’s going on.

Imagine a professional escape artist.  He has 10 years of experience performing in front of large international audiences, then one day he watches a new magician’s escape trick.  He thinks he knows how it was done so he tries to replicate it.  He chains himself up, puts on a straight jacket, takes a deep breath and locks himself in a glass box filled with water.  Moments later he realizes his understanding of how the trick was done is incomplete… Now he’s in real trouble.

That is the danger of code that works like magic.  You think you know how something works; you gain confidence that you understand it, and then it fails to do what you expected at a critical point.

What about the beginner magician?

The beginner magician can watch a card trick performed 100 times and may not be able to figure how how it works.  If they try to perform the trick themselves it’s almost guaranteed to fail.  You can not easily learn how the magic trick is done by just by watching someone else do it.

Web frameworks can be like a bag of tricks where you get to tell the professional what trick to do.  “Make this form work with AJAX” can be like asking a magician to guess which card you’re thinking of.  You have no idea how it either of those tricks work.  However, you can become very adept at wielding your bag of tricks and never learn enough to create your own.

One of the unexpected benefits of free coding daily is the chance to explore solutions to problems you have to work on that day. When you write code freely without considering the use of it there is less pressure to keep a poor implementation. It’s practice that is a way to boost your understanding of the problem before writing it for real.

There are some definite benefits to writing a solution to a problem more than once.  With each iteration you get faster and produce a better solution. When you practice and try to learn from each attempt you get better.  You gain mastery.

I came across an excellent resource last week that more people should know about.

The Architecture for Open Source Applications is a series of books that examines how some of the best software ever written is designed.

Programmers rarely get the chance to study the work of others in detail.  You learn the fundamentals in school but when it comes to experience building larger applications it’s not easy to pass on that knowledge in a single semester when writing the code for a large application may take years.

The books are inspired by how architects learn by studying the designs of other buildings. If you’re tasked with designing an office tower there are no doubt countless lessons that have been learned in the past century about dimensions, materials, efficiency, best practices, capacities for everything from floor space, water pipes, air exchangers and elevators. If architects re-invented the wheel each time they took on a new project almost all buildings would be terrible.

Software developers do often start each new project as a chance to re-invent everything themselves.  Of course this is a massive risk.  Even the best developers cannot implement the optimal solution to each an every sub problem in a large application.

Studying design patterns get you part of the way there.  Actually seeing them used in different contexts will deepen your understanding of their value.

Actually studying the great open source applications will give you inspiration to lean on when solving your own application design issues.

All software developers should read these books.

Estimating the cost of developing software is difficult.  Very difficult. That’s because with each new project there are always unknowns, mis-interpretations and assumptions that are not communicated.  If the client knew exactly what they wanted then it would just be a matter of typing it up.  They hire you because they don’t know how to do it themselves.

Agile development realizes how difficult estimations are by attempting to avoid them entirely.  Instead opting to push clients to time and materials which they have a hard time fitting into a fixed annual budgets.

Creating an estimate that accounts for uncertainty while fitting in with businesses’ existing processes is non-trivial.

It all starts by taking the project scope and breaking it into smaller pieces that are easier to estimate effort on.  There are risks even at this early first step that your estimate may be off.  Even with a great deal of thought you will likely overlook things.

Agile approaches would have you break a project up into ‘stories’ and estimate on those.  I have found that stories generally have some overlap, making it difficult to understand as a line item in an estimate.  I prefer to get down to the level of code structure.

The second thing to consider in an estimate is that with every guess you make there is some understanding of uncertainty that is not often portrayed.  Creating a login page is fairly concrete, you’ve done it 50 times and are very sure how long it will take.  Creating a custom machine learning algorithm for the application may be a bit more fuzzy. Providing a measure of uncertainty to your estimate turns your fixed estimate into a range of estimates with different probabilities.

Actually, a machine learning algorithm is useful example to explain my next point.  A good ML algorithm can take time to train and tune and a bad one can be quick. You can take more or less time and still get an algorithm that works.  There is not only an uncertainty but also some elasticity since it is unlikely that an RFP (Request for Proposals) included stipulations on the accuracy of the ML feature.  Elastic features should exist as 2 options on an estimate. 1 for basic implementation and an other that provides a better implementation.  That way the advanced option can be explicitly accepted by the client.

Next thing to consider is that not everyone’s estimates are equally good.  Some people are overly optimistic others tend to assume worst case, sometimes you have better knowledge that can help cut down the estimate while another person considers things more difficult.  By collecting estimates from many people you can average out their biases and with an open dialog hopefully team members can learn from each other.

This brings up a couple of major factors that can influence a person’s estimate:

  • Anchoring bias – A person’s estimate can be influenced if they see someone else’s estimate first.  You mitigate this by having estimates done blind.
  • Optimism bias – People tend to want to impress.  It may be to stand out from the team as better, or because they haven’t thought of some of the complexity in a solution. There is a tendency to under estimate.
  • Lack of information – All estimates are made without complete knowledge.  The client likely doesn’t have all the information either.  When you don’t have all the information there is more uncertainty in an estimate.

Anchoring bias can be dealt with through procedures that make estimates independent of each other.  Games like Planning Poker have rules to prevent anchoring bias (everyone turns over their card at the same time).

Optimism bias is more difficult to account for.  Perhaps the best (though as of yet untested in software development) approach is to use reference class forecasting to provide some insight given historical realized costs on similar tasks.  If setting up a new server has for the last 20 projects taken an average of 2 days, but the team has estimated it will take 1 day then they may be some optimism bias in the estimate.

Lack of information is accounted for by adding uncertainty to the estimate.  Although we don’t always know what we don’t know so some amount of unknowns should usually be added as a buffer in the final estimate.

Using this information hopefully you can create better more realistic estimates for your clients.  If you are a client accepting bids on an RFP you should be looking for bids that detail where there is uncertainty.

If you want to get a sense of just how crazy the world of technology is for businesses that want to make money you need look no further than Slack.

For those of you unfamiliar with Slack.  They launched a product in February 2014 which is a private communication tool for business inspired by IRC. It has quickly grown to 30,000 teams using the service sending 200 million messages per month. Now, just 8 short months after launch they have raised money at a $1B valuation.

It is astounding growth and an amazing success.  How can you copy Slack’s success?

  1. Identify and launch into an under-serviced market.  Internal business communication tools really wasn’t a product category a couple years ago and companies made due with less than ideal alternatives like email and Skype.
  2. Look for big scale market opportunities.  Chat is something that could be used by millions of businesses world-wide. A billion users is entirely possible.
  3. Minimum Viable Product – most important word is “Viable”.  It’s easy to err on the side of too minimal and end up damaging your branding.  If eager early adopters look at your product and dismiss it, it will take a long time for them to come back and re-evaluate.  Slack was in development for over a year before it launched
  4. Create a low barrier to entry.  Slack’s integration features, and free trial made it easy to convince the boss and get buy in from the team.

Of course if it was easy we’d all be billionaires!  Good Luck!

Programming is a great mix of both the creative and technical skills. Problem solving on a daily basis makes it one of the best jobs imaginable. Staying ahead of the technology curve and continuing to get better at your core skill is what differentiates an average programmer from the superb.

The three most effective ways I have found to get better at programming is

Read Books

The quality of a curated, professionally edited book written by a talented author is hands down the quickest way to learn something new. If you want to pick up a new language, framework or learn new concepts reading at least one book is a good start. I am always reading at least one technical book.

Practice Programming

I write code at a full-time job for 40 hours / week. That isn’t practice and professional experience doesn’t provide many opportunities to actually get better. Deliberate practice for software coding can come in many forms. Learning a new language? Try solving the Rosetta Code problems. I believe some of the Basic Computer Science algorithms should be committed to memory – for example – being able to code merge sort quickly because you ‘know’ it rather than having to figure it out.  Practice writing Software Design Patterns and code idioms until they become second nature and intuitive.

Peer Review

Often you don’t know what you don’t know.  In these cases having someone else review your code can really open your eyes. However, in a work environment peer review often turns into a cursory sanity check, or after a while you have managed to learn all you can from the colleagues who typically review your work.  Venture into the world of open-source by making pull-requests.  This will introduce you to a wider community of developers where you have the chance to learn some new perspectives.

With the help of unit tests, BDD, Coverage reports, Continuous Integration, and Source code control it has become easier than ever to build code that is robust against regressions while letting you branch and play with ideas quickly and without risk.  However there are times when the scale of code use in production requires a different approach to testing.

Staged rollouts through a distributed system are one way to test the waters with new code in production, but what happens when you can’t risk even 0.001% of traffic experiencing particular errors on critical pieces of code?

Tests can have the effect of making developers over-confident that their changes will not break the system.  I have seen this happen on production systems.

In the world of sales and marketing, split testing is a core concept for optimizing a sales message.  You put out multiple advertisements and measure which performs better against a set of criteria (more opt-ins, sales, links clicked etc.).  When the experiment reaches a statistical conclusion you clean out the losers and try again to beat your best performing advertisement.

I’d like to see this approach used on more than just marketing.

Split testing code would let you verify the accuracy of a new algorithm in real world usage, or measure the performance of different code paths.

People usually read about code performance benchmarks and complain that they are too simple, too specific or missing something that makes it hard to apply the conclusions of one test to another implementation. Measuring and comparing the code in your actual application scenario gives you real performance in your actual application.


open sourceThere are so many good reasons to open source code.

  • Gain contributions from the wider community
  • Contribute back for all the awesome you’ve gotten from Open Source
  • To build the status of yourself or your company
  • Attract the best programmers
  • Get public feedback on the quality of your software
  • More people will use your software
  • open source reusable components actually get reused
  • Attract clients

With those great benefits for putting more open source code out there it still seems like a hard sale. Business types don’t always see the benefit to putting time and money into creating something only to turn around and give it away. “HEY!” they say “that’s valuable intellectual property”.

As developers we know the value of producing more open source code.  It is our job to convey that message as best we can to our clients, whether they be internal or external clients.

How do you identify part of a project that is a good candidate to open source?  Here’s the best criteria:

  • It’s not unique to the core business (ie Google shouldn’t open source their search engine)
  • It should be something that has some re-useability for other people or future projects
  • Should be small and single purpose.
  • It should be easy to understand and explain
  • Ideally it should be a package (cocoapod, gem, pypi, etc), or a service

If you are given a project and asked to design it’s architecture, estimate the cost or otherwise create an implementation plan you should take a moment to consider if there is any pieces of the project that could or should be made open source.  Isolating and open sourcing should add very little relative cost (you would have had to implement the functionality anyway) and you get the benefits of publishing open source mentioned earlier.  Try to up sell your client on open sourcing parts of their project.

In many cases open sourcing a piece of a larger project may be the best business decision to make.  MBA types just won’ t have an easy time grasping that concept so it could be a hard sell, but it’s often worth pushing for.

Keep a mindful eye and suggest an open source strategy on your next project.  You, your client and the wider community all benefit when new code is open sourced.

Free coding is the practice of writing code quickly off the top of your head.  It should be done as part of a daily ritual for at least 10 uninterrupted minutes.  The goal isn’t necessarily to produce something useful or even complete.  You should strive to open the taps of originating thoughts in your head and pouring them quickly into a text editor.

Here are five reasons you should be free coding:

Continue reading

Developer productivity is a  perpetual area of improvement. Finding better tools, new abstractions, learning shortcut keys and using modern project planning is a way to continually get better at your job.  The goal of doing all that is to produce better software faster.

Choosing the right tool for the job plays a big part.

Lets say you wanted to get a web app up and running that provides a login, admin area, user accounts and authenticated REST API around some simple business form data that’s submitted by a Mobile App.  Writing that in C++ might take weeks, writing it in Go would take days and writing in Python/Django might take hours.

When deciding how to roll this service out onto some servers.  purchasing, receiving and building physical hardware could take weeks, manually building the machines on AWS could take a couple days, and deploying to a PaaS like Heroku or ElasticBeanstalk might take minutes.

There are orders of magnitude in reduction of effort required to build things when you choose the appropriate solutions.

As a developer who writes software there are several things that are easy wins for writing more code in less time:

  • Typing speed. Learning to type faster is a basic skill, expanding on that with shortcuts is awesome
  • Knowing the language and APIs.  Every time you have to Google for API references it slows you down by 10x and gives you a chance to get distracted.
  • Knowing how to do things. At a higher level than code syntax, knowing how to structure solutions, and having a good sense of what the software is supposed to do and how you are going to do it.
  • Isolation. Distractions will derail you – people, noise, tv, all get in the way of being able to focus on the problems at hand.
  • Preventing blocks. Running out of battery, losing internet, no credit card, don’t have the credentials? Getting blocked can stop you from working entirely.

The best developers really can be operating at 10x the performance of the average developer.

Getting things built quickly is not only impressive, but it is a major competitive advantage.  Getting your products to market faster and iterating on them faster is a big win for everyone.