Author Archives: Matt Warren

When researching things for a side project I dug into the advancements in services provided by Amazon AWS over the last year.  Amazon clearly has an astounding lead in the market for web service and data center technology.  The direction they are pushing things is to create a big pivot in how we all develop and deploy web applications.

If we examine the history of how development for the web has progressed it looks something like this:

  1. initially very limited programmability. static HTML only
  2. CGI scripts made it possible to execute code for a web request but was very cumbersome and adhoc
  3. PHP bridged scripts and HTML and made web applications possible
  4. Web frameworks: Django, Ruby on Rails and Java EE give developers a way to tackle larger scale problems
  5. Something new

Amazon is poking at a new way to develop server applications that inserts custom code into pre-built services.  This technology stack is still new and not very mature in terms of development methodologies but promising if it can deliver on providing scalability and low cost with ease of deployment.

By avoiding EC2 or similar raw operating system virtual machine services you can relieve yourself of a host of maintenance issues.  Scalability from zero to millions and no concern with managing security patches or hardening network access.

Lets say you want to develop a mixed web and mobile application.  With a database on DynamoDB and API hosted by the Amazon API Gateway, evented processes on Lambda and stream processing with Kenesis and block storage on S3 you have all the components needed to get an application running without configuring a linux server.

Right now the code management and deployment stories is not well sorted out.  But that seems like a problem that can be tackled.  If the efficiencies of this approach are as great as they promise to be then developers will eventually move to it.

If it does work out it could nearly make devops obsolete.

Computers have a long history of replacing the work of humans.  Before mainframes large businesses would have armies of people with punch calculators to do the work that is now in a single spreadsheet (each person performing just a cell of that sheet).

Over the past 60 years the primary productivity benefit from computers has been the ability to do our math work and implement logical rules on data.

It seems like we are finally getting to a point where computers can start to do some of the softer skills normally only done by humans.

Projects like OpenCV and Scikit-learn give mere mortal programmers the ability to write software tools that are astonishing.  There are a growing catalog of libraries for deep learning, big data and natural language processing that are at a tipping point of being easy to use and deploy.

The next generation of productivity wins will come from leveraging these new tools.  Smart email auto-replies, smarter code completion, smarter communication, and plumbing together of much more complex automations. Talking to computers as if communicating to a human and expecting it to understand.

This may be the final turning point in productivity where humans lose the ability to keep up or find a new industry to be employed.  One of the fastest growing segments of employment is being a personal trainer.  Computers so far have failed to provide a compelling replacement to person-to-person motivation at the gym.  However there is no guarantee that those jobs are not at risk given smart watches and connected equipment paired with sufficiently written software.

If that is the case then a good strategy would be to try and be on the side that takes advantage of the productivity gains and not on the side that will become unemployed because of them.  If a doctor using the latest in AI tools can see and treat 5x the number of patients as one who doesn’t use them then if you’re a doctor you should try to avoid being one of those doctors who can only examine 1/5th the number of patients.

These kind of productivity gains will take time to fully materialize but perhaps less than you expect. As they come into play more wealth will consolidate into the hands of those at the head of the curve.  It will increasingly be difficult to keep those jobs that are automatable.

It seems inevitable that this will happen and it’s up to you to make career moves that will land you on the side that stays employed.  If you’re in a position to leverage these new AI tools it will be in your personal best interest to do so.

Slack is the fastest growing startup in history, and it is starting to drive a new kind of interface for users to make things happen within their businesses.

If you remember way back the only way you could interact with a computer was through a command line.  With this interface you form very specific commands that require a manual for reference. Then with the advent of the mouse and graphical operating systems, these command line tools were mostly replaced with buttons and menus. This opened up computers to a new level of novice users, but there still was a need for manuals and books to learn how to use all these graphical applications.

Today we have two new possible interfaces for how you can make computers do what you want.  Voice systems like Siri offer a compelling next generation way to talk to a computer with your voice but the technical demands for integrating this technology is still FAR too difficult to make it work with arbitrary custom software.  The other option is bots that run in chat systems like Slack that can offer natural language interfaces that blur the boundary between what we have come to expect of computers and what the future may be like.

Slack

The other night I programmed a quick slack bot powered by the script from The Big Lebowski (one of the best movies ever).  This simple script interacts with the user in a conversational way and took less than 1 hour to implement.

This type of interaction offers a future where we can casually talk (through typing) to the computer and have it do things on our behalf.  It doesn’t require a manual to know how to specify commands.

This seems like a step in the right direction towards a broad range of tools that work in a more natural way.  Ultimately replacing the text input with voice will bring us to a Star Trek style future. But until that time comes we will be continuing to develop frameworks that allow developers to create interesting tools that know how to interact with users naturally.

These bots can trigger plenty of things that might not have been possible with other interfaces.

>> matt: hey @meeting_assistant, start a meeting with @jack and @jill

>> meeting_assistant: ok @matt, join the Google hangout <here>, I’ll keep meeting minutes of anything you type in here.

>> matt: @jack can you contact the client about that thing at that place tomorrow?

>> meeting_assistant: @jack I added a reminder to your calendar for this ^^

>> matt: @meeting_assistant, all done.

>> meeting_assistant: ok, I’ll email the meeting minutes to everyone.

This sort of theoretical chat bot is now entirely possible and it may just usher in a new age of business communications.  It’s now easy to forego the large enterprise application development cycles and instead focus on small text based services that can easily be deployed.

The future of this kind of interaction may create a new opportunities to reduce costs and streamline business functions.  By triggering messages based on external events it’s possible to prompt employees to their maintain their workflows, or reward them automatically:

>> time_bot: @matt, you haven’t recorded your time for yesterday.  please update your timesheet.

>> overtime_bot: @matt, thanks for staying late, but it’s time to get back to your family

>> kpi_bot: @matt, your performance has been exemplary this week, we’re sending you an Amazon gift card as a thank you.  Keep up the good work.

I think this could be a big shift in how businesses run.  And companies that take advantage of this next generation technology stand to gain a tremendous competitive advantage in the coming years.

I recently had an experience where I was given the same programming problem as a handful of other software developers.  My advantage was that I was using Python while the other people were forced to write their solution in Java or C.

The difference in the amount of time it took to solve the same problem with a different language was striking.  Most other people completed in about the 45 minute mark whereas I had finished in just 15 minutes.  An average of 3x less time.  That’s a massively significant difference in developer performance that comes primarily from using languages that aim to be readable and focused on developer productivity.

I’d always known that Python was a productive language but that was the first time that I had experienced first hand just how beneficial it can be to switch from one platform to another.

Imagine taking a 3 month project and complete it in just 1.  That’s a massive savings; enough to change a business idea from impractical to highly lucrative.

One thing I’ve learned over the last couple of years is that with the right approach and the right tools you can make difficult application development accessible to even small budgets.  Sometimes it requires creative thinking and accepting unconventional solutions.

I’ve worked on $1M projects that could have been MVP’d for 20 cents on the dollar by choosing different technology and different developers. It’s quite shocking to see just how variable it can be to develop the same software with different tools.

If there’s a lesson in this it’s: Take care when choosing the people and technology.  The right choice can save you a tremendous amount of time and money.

Productivity is closely related to skills and talent so it’s been something that I have a keen interested in.  The actions you take to get better at what you do have fairly important role in your wealth and happiness.

I recently read the book “Talent is Overrated” by Geoff Colvin.  It was insightful in a lot of ways and built upon ideas that I had myself about what it takes to reach for the pinnacle of success.

Geoff busts a number of myths about successful people.

  • High IQ does not correlate to significantly more success
  • Fantastic memory is a skill you develop
  • “Deliberate Practice” is the (only) key to improvement
  • Experience in many cases correlates to lower job performance
  • Child prodigies rarely maintain their lead into adulthood

In a handful of skill sets there is a very clear framework for practice.  Music, athletics and chess have very well understood methods for improvement.  You start early, practice, learn and exercise until your skill improves. Stick with it long enough and it’s possible to reach world class standards.

With other domains of knowledge the frameworks for practice are not as well defined.  How do you practice software programming, project management or reading x-rays?

If deliberate practice is the determining factor in skill acquisition then how can we make sure that the education system supports it, and society builds on it to influence corporate decisions in an ongoing way?

The term “deliberate practice” is specific.  Simply working 9-5 at work related tasks where you’re expected to produce output is not considered practice.  Deliberate practice is hard by design, it is a process to continually push you slightly past your current limits, repeatably, and with feedback.

Hitting a bucket of golf balls at the driving range is not practice, but it provides the illusion of practice.  Without a coach to identify and provide feedback on each swing it will be very difficult to improve your skill.  Even Tiger Woods, at the top of his game, had a coach.

To answer the question for something close to my heart, how do you deliberately practice software development, I believe the answer lies in either finding someone better than you who can review or pair program with you who also understands how to coach, or by finding superior open source code to study and reproduce.

One of the examples from the book was how Benjamin Franklin learned to write. He picked articles from Spectator magazine that he wanted to emulate and he would take notes about the article.  After a few days when he had forgotten the text he would open his notes and try to re-create the original article. Then by comparing his against the original he could identify what he could have done better.  In the absence of a coach, this seems like a fairly ingenious way to get immediate feedback on your practice.

It would be good to test this sort of approach with programming skills.  It’s a concept I have started to test.  As with all practice, it’s is a long effort before the benefits are expected.

Over the last couple weeks I’ve had a growing concern that my office is just not functional any more.  A standing desk in a room pulling double duty as a spare bedroom.

So over the last week I have pulled out all the office equipment and turned the room back into just a bedroom.  The experience made me realize just how much clutter there is everywhere and it all creates distraction.

The white tennis shoes scenario.

There’s a classic book for writers called “The Artists Way”.  One of the anecdotes is about a pair of white tennis shoes.  Imagine for a moment that you have an immense project in front of you – months worth of work.  And in the dead of winter with a foot of snow outside you sit at your desk staring at your computer monitor.  Then, from the corner of your eye you see your white tennis shoes tucked in the corner. There’s a little bit of mud scuffed into the toe;  you look back at your screen.

Moments later there is a nagging feeling, you must clean those shoes.  Never mind that you won’t be able to play tennis for months.  This must be done now. “It’ll only take a few minutes” you tell yourself.

Several hours later you’ve cleaned the whole room and accomplished none of your work goals for the day.

A messy office is full of white tennis shoes and as a result I haven’t been getting much work done for what feels like forever.  To address this I’m taking the unusual approach of getting rid of an office entirely.  I’m focusing on getting my mobile office in better shape so I can easily re-locate to a coffee shop, the bedroom or the kitchen table to get work done.  By just moving to a place that has limited distractions I’m hoping I can stay more focused and get more done.

The new generation of Apple TV was announced earlier this month and I believe it has the capability of kick-starting a new generation of TV Apps that will bring the personal aspects of Mobile experience to the living room.

Apple has a tendency of getting down to the core problem and working hard on it until it’s solved.  with TV the competitors simply took android or Linux or Windows and put it on a TV screen.  It ignored the critical differences with a TV that make it difficult to simply apply computer or mobile experiences unmodified.

  1. mouse input doesn’t work – apps that are designed to work with a mouse cannot function well with anything you can put into a remote.
  2. keyboards are not welcome in the living room.  nobody wants a full size keyboard on their coffee table, and micro keyboards all suck
  3. Traditional TV remotes don’t offer enough to accommodate complex interfaces.
  4. Video game controllers are good for people who game but too daunting for non-gamers

Apple brings a couple key things to the table with the new Apple TV that I think will be enough to capture the market and dramatically change how people watch their TV.

  1. Dedicated App Store with TV specifically designed apps that are reviewed by Apple for quality
  2. Back-end infrastructure in place for payments, subscriptions, in app purchases, millions of credit card accounts already registered
  3. An army of developers who already know the languages and libraries required to develop apps for the platform
  4. Siri to help out with some very clever ways to navigate content
  5. putting a Trackpad on the remote will make input and navigation much more natural and faster than arrow keys

I think it’s safe to say that a TV App Store with 1,000,000 apps will dramatically change the way we consume content on our TVs.  Imagine Sesame Street as interactive games that demand kids actually originate thoughts rather that passively consume them.  If Siri can help you answer anything from what is 2+3 to how does quantum teleportation work? or save Carrie’s shoes here to my pinterest #wantthese.

Transitioning TV from a passive to an interactive entertainment backed by the variety from thousands of developers pushing out their best ideas of what could be cool on TV is going to be a recipe for disruption.

I suspect there will be a killer app and it may not be a video game or related to traditional TV content at all.  It might be something only possible on an open platform where indie developers are free to create and publish their craziest ideas.

In an age where we expect everything quickly it seems that it can be harder to have the patience to endure long term investments.

People are career hopping between companies and entire job types more today than at any time in history.  Employees who want to progress up the ranks find that a quicker route is to job hop.  Ambitious employees are not content to put in 10 years at a company before getting a promotion.

Investors also look for quick gains.  Few are willing to park their money for multiple years in order to get rewarded with modest gains. We look for the quick buck.  Day trading to make 1000’s of dollars in hours instead of months.

This psychology has repercussions in our endeavours.  It becomes hard to focus for years on a blog to build an audience or create a piece of software in our spare time, or start a business.  These long term goals take time and time is the one thing that we too impatient to wait for.

However, being conscious of this is the first step to being empowered enough to account for our natural tendencies and make long term decisions that are in our best interest.

 

A project I have been working on for the past couple of months has suffered from a critical planning error that early on has resulted in time and budget overruns as well as a product that has performance and stability issues.  It’s all because the decision makers at the top decided to take on risky technology and design choices anticipating scalability problems before they even have scale.

One thing that I have learned over the years is that you can go a long way on the basics.  MySQL or PostgreSQL will scale very well for almost all use cases you are likely to be developing for.  In very few cases is the horizontal scalability of NoSQL options worth the complexity it brings.  Building out a distributed clustered environment from the start is a sure way to add unnecessary complexity to every subsequent technical decision.

Now the project is over budget and behind schedule.  Any work that needs to be done has to deal with an architecture that adds complexity to every software decision and makes deployments more error prone and time consuming.

Lesson learned I guess..  An MVP release should really focus on the keeping things as simple as possible for as long as possible.  For a $1M project, a 50% cost/time overrun is a lot of money.  Especially when a simpler architecture might have been 50% below the initial budgeted amount.

At this point we all know that being a good multitasker is a myth.  The time it takes to context switch between projects can vastly wipe-out any perceived productivity.

This has become more apparent to me recently as I have been asked to take on more work, or have taken on more for myself.  As a result annual goals fall behind, projects make slow progress and planning overhead rises.  This has knock on effects to perceived expertise – the amount of forward movement on a particular project is less in real-time than hoped.  When a month passes by with only a week of progress things are intuitively slow and perceived value drops.

It’s important to say “No” to incoming requests and limit what is on your plate at any one time.  This actually can have a positive impact on your perceived value.  Saying “Yes” and failing or being late is far worse than saying “No” and leaving the impression that you have more important work to do.

Focus is critical to both your own stress management, your external perceived value, and your productivity. Strive for only what is important, say “no” more often, and you will benefit.