Author Archives: Matt Warren

The morning is probably the best time of day to get things done for yourself before work and stress start to derail your day.  When you’re trying to accomplish something big finding a way to take advantage of these early morning hours can really ramp up your productivity.

The quiet morning hours can be your time to focus on what really matters most for your long terms goals.

What nearly always seems to happen is that at the end of a long day you are too exhausted to go to the gym, your too stressed to focus on things, or you’ve been convinced to spend an extra few hours at the office to finish up.  Your own priorities slip down the list.

A day has only 24 hours and every bit of time you give to something else is time you can’t take back later to do things for yourself.  If you want to make sure your goals get hit you need to block out time for yourself.

In financial planning one of the biggest tips is to always pay yourself first.  When you leave yourself whatever is left over after paying bills, and other expenses there is almost always less than you need to save if you want to become wealthy or retire.  Paying yourself first helps you reach these long term goals.

The same applies to time.  If you want to build wealth out of time rather than money you should use time for yourself first before you give it to other’s.

Creating a morning routine, and making it stick has in my case been successful when I’m truly excited to get out of the bed in the morning, I have everything ready to go, and no excuses preventing me from getting up.

One of the best things you can do is to prepare the night before for the morning.  Have your work ready, water ready to drink, clothing taken out and coffee maker scheduled.  The easier and quicker it is to get from bed to fed and getting things done the more likely you are to do it.

Here’s what you need to do:

  1. Figure out what you what to accomplish in the morning.
  2. Get everything ready, so you can do that as soon as you get up
  3. Iterate.  Find ways to make getting up easier, until it is easy

The theory for motivation has been best described by Maslow’s hierarchy of needs. Peak motivation can be achieved when you become self-actualized – internally self motivated, however to get there you need to fulfil your lower level needs first.

Maslows Hierarchy Of Needs
Maslows Hierarchy Of Needs

At each level of this pyramid you will find factors that might be motivating you.  Working up from the base starts with the physical needs – food, housing etc that we need.  If you don’t have these needs met then you would be strongly motivated to satiate them.  Note however, that they are not good motivators to achieve great things in life.  You will never compose an orchestral masterpiece if you can’t afford food and are living on the street.

The next level is Safety, which includes job safety and personal safety.  When you live in fear of not knowing how long you’ll be employed or if the city you live in is safe enough then you will be motivated to mitigate that risk.

Love and Belonging.  Humans have emotional needs to have friends, and family connections that are strong.  Building and maintaining these personal connections can be a strong motivator.

The need for self-esteem and self-respect is important.  People with low self-esteem may be motivated to impress others in order to gain respect.  However, your perception of how others perceive you is not always accurate, and the drive to impress is not an effective way to fulfill this need.  Self-respect comes from within.

At the top of the pyramid is self-actualization.  This is the need to reach your personal full potential. The more time and focus you have to devote to this need the more you are likely to achieve in life.  Someone who is fully self-actualized has all their lower needs met, and can devote all their efforts at pushing their own potential.

Knowing where your needs are is the key to identifying you motivations.

Over the last couple weeks I have been getting acquainted with using Ansible to provision and deploy servers. One nice thing with using Ansible over a tool like fabric for deployment is that everything from provisioning, initializing, updating, management, and application deployment can be done with a minimal amount of code.

There are tips I learned as I got started with Ansible:

  1. install ansible with pip into a virtualenv.  This seemed to fix issues I ran into when having it installed globally on my computer.
  2. create a new directory to work with – it will have all the templates for nginx, supervisord, roles, playbooks, and server lists in it
  3. create an inventory file that lists all the servers you will manage
  4. split things into multiple playbooks – I have a webserver, database server and deploy playbook. having the deploy playbook separate keeps it simple and it will run quickly.
  5. the roles found on ansible-galaxy are almost never exactly what you want, even if they are you’ll likely have to read through the templates to find out what variables can be used to customize them.  In almost all cases it’s good to clone the role into your working directory’s roles subfolder so they are easily accessible and editable.
  6. the idempotent property of ansible playbooks make it easy to procedurally build up your playbook.
  7. it’s often easier to have an open ssh connection to manually see what’s changing on the server, make sure templates are rendering correctly etc.

my working directory looks like this for a simple single server Flask webapp.

digital_ocean

For my deploy playbook things couldn’t be simplier:

---
- hosts: webservers
  vars:
    app_name: awesomeapp
    repo_url: https://github.com/mfwarren/secretproject.git
    repo_remote: origin
    repo_version: master
    webapps_dir: /srv/webapps
    virtualenv_root: /srv/webapps/pyenv/versions/latest_v3
  tasks:
    - name: git pull project
      git: repo={{repo_url}} dest={{webapps_dir}}/{{app_name}} version=master
      notify:
        - restart app
    - pip: requirements={{webapps_dir}}/{{app_name}}/requirements.txt virtualenv=latest_v3 executable={{virtualenv_root}}/bin/pip3
    - shell: "{{virtualenv_root}}/bin/python3 manage.py deploy"
      args:
        chdir: "{{webapps_dir}}/{{app_name}}"
  handlers:
    - name: restart app
      supervisorctl: name={{app_name}} state=restarted

the tasks are very easy to follow, git pull the latest changes, install requirements that might have changed, run the ‘deploy’ command on the app which does DB migrations, and finally kick supervisord to restart the app.

On the command-line my process for executing this deploy looks like:

ansible-playbook deploy.yml -i ansible_hosts

Ansible is a tool for power users. It would be quite difficult to use without experience setting up the servers manually. Unlike Heroku or another PaaS, it will take several hours to learn and configure your first server through Ansible. However that onetime cost should pay for itself with the additional control you have, and server cost savings.

I certainly would love to see tools created to analyze your project and create a set of playbooks and templates which would create a production quality server setup, similar to how Heroku’s Buildpacks can figure out how to get a Django/Rails/Node.js app running.

Everyone who does web development will probably find themselves doing devOps tasks at some point.  I keep an eye on servers hosting several different projects at different cloud providers and some private servers.  Until recently management of those servers wasn’t difficult enough to warrant using tools to deal with things.

Ansible is a commandline tool that allows you to manage your servers.  Lets say like me you want to introduce it into an environment with servers already in place.  Here’s how you can get started.

Install the Ansible commandline:

brew install ansible

Create a hosts file (file that lists all the servers you want to manage):

ansible_hosts:

[ec2]
ec2-.compute-1.amazonaws.com  ansible_ssh_private_key_file=/PATH/TO/key.pem  ansible_ssh_user=ec2-user
ec2-.compute-1.amazonaws.com  ansible_ssh_private_key_file=/PATH/TO/key.pem  ansible_ssh_user=ec2-user

Ping all the servers to see if they respond to ssh:

ansible ec2 -i ansible_hosts -m ping

Ansible comes with a large library of low level ‘modules’ for doing interesting things on your servers. You can call simple things right from the command line as in the previous example (uses the ping module). If you want to upgrade openssl package on all your (Ubuntu) servers:

ansible ec2 -i ansible_hosts -m apt -a "name=openssl update_cache=yes" --sudo

or upgrade all updates to a CentOS server:

ansible ec2 -i ansible_hosts -m yum -a "name=* state=latest" --sudo

There are over 200 modules built into the core of Ansible to make interacting with lots of different programs easy – git, shell commands, postgres databases, supervisorctl, django, gem packages. Doing a git checkout of your application across a pool of web servers can be done with a single command line.

If that was all you could do with Ansible I’d be happy. But that’s really just scratching the surface. Building on these low level modules is what Ansible calls ‘roles’. Roles encompass a series of commands to do a bigger task – like install and configure nginx. You can tie multiple roles together into a ‘playbook’ that can define how to build out your entire infrastructure.

In my next post I’ll cover how to go from zero to fully deployed web application with Ansible.

Expanding on the thoughts in my last blog post about measuring opposing indicators I wanted to explore things from a different perspective.

There is a growing community of people that measure and record as much as they possibly can about themselves.  This quantified Self movement is predominantly focused on health related aspects exemplified by the growing market for activity trackers like FitBit.  The more information you have to track yourself the more you are able to identify longer term trends that would otherwise be twisted by your own fallible perception.

Programmers strangely do not have much going on for measuring themselves.  Even though it would be quite easy to glean information from the coding output from any programmer.

I would be quite interested to see a collection of stats about myself.  A daily digest of basic productivity, and charts of longer term trends would be motivating and insightful.  Would lines of code per day increase as I got better at a new language/framework?  Would there be plateaus in my productivity? Could I identify the impact of distractions? How much more productive am I working from home vs. at the office?  There are so many questions that would have valuable answers for me personally in my quest to improve.

While no measurement is going to be perfect, by measuring myself against myself, for personal insight provides some incentive not to cheat and keeps the relative performance as an informative metric.

GitHub provides a decent API which could be used to dive in and gather this information.  It would be interesting to launch a service that could mine through a user’s github account to generate a detailed productivity report.

Unfortunately another project I don’t have the time to build myself.  If someone else builds it though. I would surely be a paying customer.

All software developers know that measuring a developer’s productivity by counting lines of code written is not an effective way to measure their output.  The correlation between the technical difficulty of a problem and the number of lines of code is not always 1:1.  Which means that one developer can write 10 lines in a day and be very accomplished, while another could copy/paste a solution of 1000+ lines in a day which was trivial.

On the factory floor managers always should be measuring opposing indicators.  If you’re measuring the volume of output – you also need to measure the quality of output in order to prevent creating an unbounded method for taking advantage.

For example.  If a manager of Intel processor factory production line focused solely on the number of chips produced each day, and cancelled all quality control checks then production output would likely skyrocket.  In short order, however, the defects would be found by customers, the business would suffer.  On the other extreme, if the manager focused solely on quality and checked every aspect of each chip produced then costs would skyrocket and output would crash.

The problem with measuring a software developer by the number of lines of code they produce is that there isn’t an opposing measurement of the quality/difficulty of those lines.  Gauging the quality or difficulty of the produced code would likely require peer review, and for that peer to grade the code.  Nobody wants to be the ass that gives their co-workers D- grades on their code so it becomes difficult to get honest scores.

In my opinion, counting anything is better than counting nothing.  If Bill suddenly goes from committing 100+ lines of code per day down to 10 lines per day it may indicate something that a manager should investigate.  If you weren’t measuring output then it would be impossible to know about or fix anything that Bill was having trouble.

Authors measure themselves by pages written per day.  Writing 10 pages of crap is better than ending the day without having written anything. At least once things are on paper they can be communicated and others can help edit things to find the gems inside.

Programmers often take something for granted that would make them significantly better at their job.  No it’s not communication skills, or management.  The biggest differentiator between the best and average programmer is how often they have to refer to google to solve a problem.

Any programmer who knows the basics of programming in a C like language feels like they’re competent enough to write Javascript – even if they only have to program it occasionally.  What ends up happening is they stall on every other line of code to crawl stack overflow for solutions.  Despite their lack of skills they can still write complex code and get things done.

No doubt that Google and Stack Overflow are awesome resources that make it easier to write things you don’t really have the knowledge to do off the top of your head.  Each detour to the search engines eats time.  It takes 10x more time to find a good piece of code on the internet than if the programmer knows how to write it already.

The skill that most programmers overlook is memorizing the APIs and languages they use.

When you take away all the crutches we use to help write code by writing with pen and paper, without the help of reference material gaps in your knowledge become brutally apparent.  Try this from time to time.

Most programmers I know seem to think that picking up a new language or framework can be done in just a few weeks.  My experience is that unless they take deliberate time to read and learn and build expertise the typical result is getting to a passible skill level and then plateauing.

Do not underestimate the value of depth of knowledge.

ToolboxDo you want to write code faster, with fewer bugs, and feel confident that what you write is good?

Over the years, programming has evolved, and many new platforms and tools have been created. Most of the tools aren’t very useful, but a few are.

Here are the 11 tools that have helped make my programming a bit easier and more successful:

Tool #1: Linting

Every language these days has a selection of Linting tools.  These are programs that parse your code to detect and flag best practices.  Linting tools enforce consistency of code style among all those in your team, can detect complexity, unused variables, naming conventions, or a long list of other infractions that help you to keep your code clean.

A side benefit from Linting tools is that they can hint at things which may just be general improvements.  Enforcing Javascript semi-colons, suggesting modernization of Ruby code or helping keep Python code PEP8 compliant can often trigger you to think twice and maybe re-factor to improve your code structure.

If you’re code environment doesn’t have linting built in, get a better development environment, or install a plugin. Linting is a great tool for improving your programming skills.

Tool #2: Dash

Google and Stack Overflow are great resources for solving quick questions you have about something you need to do but having a local copy of documentation to reference can speed you up tremendously.  Dash is a Mac App that lets you search through documentation for various languages and frameworks quickly and easily.  It copies them locally so it works when you’re offline too.

Everytime you search online for an answer to a question there are two things that can go wrong:

  1. You get distracted reading articles, browsing things and loose time
  2. You get the answer you want without enough knowledge to understand it, copy/paste and promptly forget how it works.

Reading real documentation often forces you to learn something new.  Well written documentation will explain things enough to give you a firmer foundation to help you remember it the next time you encounter similar questions.

Tool #3: A Solid Text Editor/IDE

I have been using Atom for the last couple months as a coding editor.  I’ve always personally felt that IDEs can often do too much which makes me uncomfortable.  Which ever side of the fence you’re on, one thing is clear.  You need to get yourself a great editor.

This is the primary tool you use when coding – don’t be afraid to spend a little bit of money to make your life better.  Take the time to truly master this tool.  Write your own plugins, customize the shortcuts, tweak the theme, read a book!

Tool #4: Vim

Yes, vim is a text editor, but regardless of your editor of choice you should also know how to use vim.  Especially if you work on servers.  Vim is installed by default on just about every Linux machine, so it’s the workhorse that is very nearly always available for you to use.

Tool #5: Git

There are many methods for managing source code versions. Git is the one you should be using.  CVS and SVN are long dead, Mercurial and Baazar lost out to git.  Even if you just are coding locally, without any other developers git is a gift that will someday save your ass.  There is really no excuse not to have your code under source control.

If you want a GUI for interacting with git – checkout SourceTree.

Tool #6: Commandline

A master of the commandline can accomplish a lot of things with relative ease.  One of the nice things about a commandline interface is that if you know how to do something on the command line (as opposed to in a GUI) you know how to automate it or turn it into a script without any additional thought.  With a GUI you know how to do something by moving and clicking the mouse – and it’s unlikely you’ll every investigate how to do that task any faster even if you have to perform it 1000 times.

Tool #7: GitHub

You’re using git right?  GitHub makes git better in several ways.  By adding Issues, Wikis, Pages, you can plan your Todos and publish your documentation.  You can easily give access to repos to new people when they join the team, or make things public so anyone can fork your project.  Use Github webhooks to plug in and automate your own workflows.

Have a public face to yourself and your company. Github is the first place I look when checking out a potential new hire.  Your github profile is the modern extension of your resume.

Tool #8: Newsletters

K, this isn’t really a tool, but I think it’s important.  Reading the various Google+ Pages, Reddit subreddits, Twitter, and blogs for the languages, tools and frameworks you use, is simply too much noise to reasonably parse.  I suggest signing up for a weekly newsletter instead.  A much more reasonable weekly digest of the most important new things will keep you up to date, without sucking up all your time.

Tool #9: Books

Books are often underrated by software developers.  Unlike blog posts which are often written quickly and published in the first draft.  Books are often written by multiple authors, reviewed by a panel of experts and edited by professionals.  These extra filters helps to lift up the quality of content in a book much higher than you find laying around on the web.

If knowledge a $20 book gives you saves you 10 minutes then it’s paid for itself.  It’s much more likely that a good book will save you days.  A great book could help you double your efficiency.

Tool #10: Continuous Integration/Unittests

I have found that different languages and tools necessitate different levels of unit testing.  Python is relatively explicit and APIs are generally consistent – python code usually behaves the way you expect it to.  Ruby syntax on the other hand is prone to ambiguity, has multiple syntaxes for the same things, is poorly documented and often implicit in how things work.  Even after over a year of full time Ruby development under my belt I trust that any untested code is broken.

Continuous Integration runs your tests after every commit, or every change you make locally.  By running tests often you can catch regressions quickly before they become an embarrassing bug.

I recommend using a web service like Travis-CI.  Setting up your own CI Server can be a bit of a pain.

Tool #11: Practice

If you hired a coach to teach you how to play tennis, you expect to run drills, learn how to grip, swing, stand and step. You’d hit thousands of balls until the mechanics of how to hold the racket, swing and step become so instinctual that they can be performed without thought.  You would practice several times per week, for 1-2 hours each time just performing basic moves.  It would be months before you actually are ready to play a game.

Nobody practices anymore.

I believe that deliberate practice is what differentiates an average developer from a 10x developer.  Practice with the languages, frameworks and libraries you use and the tools you write with.  Every API that you memorize will add up to hours of saved time searching Stack Overflow.

Conclusion

Writing code doesn’t have to be hard. There are a ton of free tools out there that can help you write better code faster.

The 11 tools I mentioned above are the main ones I use on my projects. Give them a shot. They’ll change your coding experience.

What other tools do you use when coding?

Something has happened in the manufacturing world that is nearly irreparable.  As America moved production to China and downsized factories to realize the profit gains from low wage Chinese workers it also decimated an entire skill set from the national tool belt.

When a factory tools up a new line to produce a new iPhone, sneakers or toy it is the job of a skilled tool maker to decide what custom tools can be designed to make things efficiently.  These may be custom 1 of a kind drill bits, platforms, moulds, or machinery that someone with a creative and technical skill-set needs to design and create.

When America outsourced it’s factories it also lost these very valuable skills.  Now, as a result of that loss of knowledge spinning up a new factory in America is a very difficult task to accomplish.  That loss will take a long time to recover from.

In the world of software development, abstraction is often a panacea.  Abstraction has created a similar migration of technical skill sets over the last few decades.  We abstract out the complexities with layers upon layers of helpers, protocols, frameworks and libraries.  These extra abstractions can boost our productivity tremendously so long as we don’t encounter a use case that isn’t handled by our chosen abstraction technology.

The danger when you completely ignore how those abstractions work is that the skills and knowledge it took to create them die out as well.  Who are the tool makers?

I have found it incredibly useful when trying to understand a new tool to have a solid grasp of the underlying technology. Knowing how to build a HTTP service at the TCP level (and from time to time actually doing it) makes it easier to reason through issues and push the limits of a framework like Ruby on Rails.

Software developers everywhere should spend just a little more time writing the tools they need to do things quicker in the future.  It is investments in the tooling that has the chance to compound to make future work exponentially easier.

Create stand alone services that can be leveraged on multiple projects, write code generators, and libraries. Create your own frameworks.

The style of programming required for creating a great re-usable library is very different from that used when doing a business application.  Good, flexible code often relies more on using the dynamic capabilities of a language.  You’ll often find yourself needing to use meta classes, introspection and other fun constructs.  The people that know how to wield these tools are going to be better at debugging, see opportunities to improve code, and create their own tools.

My plea to software developers everywhere: create your own tools.  Know how to create the tools you use, and how to use the tools you create.

This last month or so has been a hell of a busy time for me.  Working long hours, with a 1 year old at home who needs attention while trying to squeeze in social events, exercise, sleep, reading and personal projects is a lot to juggle.

How have I managed to rock a streak of 53 consecutive days with open source github activity, learned Flask, read 3 books this month and made progress on several side projects?

The Github streak never would have happened if I didn’t have a script to email me every hour after 5pm that I don’t have activity on my account.  This simple nag makes sure I don’t go to bed and forget to write some code each day.

Thinking of things to code each day has pushed me to explore new ideas, practice concepts, study APIs and do tutorials. In retrospect it is astounding just how much interesting stuff you can get done when you do a little bit every day.

A simple nag will actually make you find the time when you don’t have it.  Several times per week I find myself up later than I would have been otherwise trying to write just a couple more lines of code.  Once you start on an idea it’s not easy to let it go unfinished; it’s hard to commit code that won’t even compile.

Activity has spiralled off from this daily coding.  A couple days of hacking on some stock market analysis algorithms got me checking Amazon for books,   which in turn got me to completely overhaul my personal finances.  On another tangent, trying to think of ideas for things lead me to a tutorial on Flask and more books on that subject.

Want to get more done?  I’ve improved on the Jerry Seinfeld Productivity Secret with a simple script to make sure I don’t destroy my streak by being absent minded.  The knock on and compounding effects from all this daily activity has been unexpected and amazing.