CI/CD — A Quick Aside

So what is CI/CD, what does it do for you, and do you need it?

I did a quick search on the interwebz before deciding to write this and what I found at a glance fell into 2 categories:

People selling a service
People hyping a trend

Since my target audience is, largely, a solo dev who doesn’t really have the context needed to parse either of those kinds of posts, I wanted to break it down a little more succinctly.

But First, Some Vocab:

CI (continuous integration):

What it Is: The act of integrating the code being worked on into the main code base on an on-going basis, rather than waiting until a feature is complete and you’re ready to release it. Typically, merging code or pushing local changes triggers a service (a CI system) that will pull the new code, build it, run unit tests, and let you know if you managed to break your application with the latest changes. Code repos like GitHub can be configured to block pull requests that have a failing CI system build. Often, the CI system will archive the built application so it can be used later in deployment.

Problem it Solves: At root, it helps you limit the damage you can do to your code at one time. Rather than checking out a feature branch, working on it for weeks, then trying to merge it back into master and finding out your feature code is incompatible with the master branch with 231 merge conflicts (or failed tests), you can catch issues almost immediately–when they’re much easier to fix. With multiple people in a code base, this gets even more important; everyone gets aligned far more often and reduces the amount of effort needed to keep things moving in the same direction.

CD (continuous delivery/deployment):

Is it Delivery or Deployment? It depends partly on who you’re talking to (I personally don’t view them as different things and will use them interchangeably) and what the context is. If my code is going to some other department that will do the rest of the dev-opsy stuff, it makes more sense to call it continuous delivery–you’re delivering it to someone. But, if your team is responsible for getting it to the users via a webservice or some other process, then it’s more of a deployment of the software. Context is the key, friends.

What it Is: Either way, this is the process of continually putting finished software where it needs to go. This could be a staging environment, a testing environment, or directly to the users if you’re a company like Facebook (or you’re just trying to show off to family like me). A common approach is the CD system will wait for the CI process to successfully complete, and then once it knows the code is good it will either take the output of the CI process (generally referred to as a “build artifact”) and deploy it based on the configured settings, or it creates a whole new artifact for the deployment.

Problem it Solves: In the not-so-distant past, once software shipped it was done, ready or not. Think about all the games you’ve played with crazy bugs and glitches in them that you just don’t see in modern games–games these days can get patched without inconveniencing player or developer. That’s continuous deployment at work. Finish some code, ship some code. It also lets you put a basic application in front of someone, say a client, to get instant feedback–and then just keep adding functionality.

Is It For You?

So if you’re new enough to all this, is CI/CD something worth setting up? I mean, it depends…but my short answer is yes.

It’s not as arduous to setup as it may sound, especially when you look at guides put together by people trying to sell you things. If you do it as one of the first steps in your process, it’s fairly easy to get done–and it really is a velocity accelerator. You won’t get caught by surprise trying to merge in new features (“When did THOSE tests start failing?”) and don’t have to fumble around moving files in order to make sure things work or to show off a weekend of work.

Of course, there’s also the fact that those are the kind of skills that separate someone who can write code and someone who can delivery value to an employer or client. While writing software is a challenging a rewarding pastime, the best program in the world isn’t really any good until someone can put it in front of users.

August 8, 2019 1

Battleship Skeleton Code

The two forward 16 gun batteries of the USS Massachussetts

But first:

I’m assuming that you know the basics on how to create a code base in your language of choice, in your IDE or text editor of choice. This series is aimed at folks who can stand up a console and/or web app on their own, with really high-level examples in C#. The goal of the series is to help you start leveraging CI/CD automation, not learn how to set up your dev environment.

I’m also assuming you’re working with Github. There are plenty of other ways to do git, and you may be able to use your service of choice in some places–but that’s on you, my guide will be based on Github. If you want a run down on this before we get rolling, this is a pretty solid article.

Okay, so why skeleton code? What does that even mean? Let’s get into it.

K.I.S.S.

If your goal is to write some code as a kata, and don’t really intend on deploying it anywhere or configuring a pipeline, you don’t really need to be simple. You can go hog wild, and test drive as much as you want as fast as you’re able.

But if, as this whole exercise states, the goal is to set up a project that can continuously integrate and deploy by automated means, the absolute last thing you want to do is put together a bunch of code with moving parts. The more moving parts to your code, the more things that can go wrong. Add in all the moving parts to configuring a CI/CD pipeline, and you’ll very soon wish you had fewer things to troubleshoot.

Okay, How Simple?

For the “business logic” you should have one small passing test. How that is architected is up to you–I’m doing this in C# and you can eyeball the repo for yourself. The standard convention is a class library project and a unit test project, with one test class for each class in the library project.

You’ll also want a super bare-bones web app. Some languages/frameworks (like ASP.NET Core) build quite a bit into their “default” templates, so chances are pretty good you can get a basic “This is my webpage” type app without having to write any code at all. In my case, I changed the HTML on the index page just to amuse myself.

And that’s it. The webpage should appear when you run it locally, and the tests should pass when you run them. Otherwise, you don’t actually want your code to do anything yet. The less that can go wrong with your code, the easier it will be to setup the pipeline.

Once you have that code set up, with tests passing and a webpage viewable, go ahead and push it up to your repo. Then you’ll be ready for the next step, linking your repo to Travis-CI.

August 4, 2019 0

The Almighty Card Board

A Trello list of cards showing several color coded tasks

If you’re a professional developer, and your team isn’t using a ticket system, you’re probably familiar with card boards. You may even be a Trello wizard–I personally have spent more time in Microsoft’s Team Foundation (now called Azure DevOps), but it’s the same idea. A way to track what needs to be done and how long it’s taking.

Since this is a (relatively) simple one-off project, I just threw together a Trello board. Here’s the 800-mile-view:

My backlog is the total sum of what I want to get done/cover in my post series. I have code items (labeled in orange), infrastructure items (labeled in green), and posts to write (labeled in yellow). I have them roughly grouped in order by things I want to do for a specific post.

Current Effort is what might be called a sprint–it’s the group of cards I think are obtainable to get done in the next week or two. To continue my trip analogy this is just the next leg of the trip–maybe how much ground I want to cover before stopping for dinner. It would be the point where I would feel comfortable putting in a pull request, a point where if need be I could say “I have to put this aside for three months” and be able to pick it up as a complete module of the larger project. The acceptance criteria for each “effort” is to have one or more blog posts scheduled.

In Prog is just that–in progress. Stuff that I’m still working on, since I anticipate doing little bits throughout the week.

Done should be another self-explanatory item. It’s all cards I’ve completed–not just during one effort, but over the course of the whole project.

My first step was to fill the backlog. Everything I needed. For instance, I knew I needed a simple code library to provide some sort of functionality. I also knew there’d need to be some sort of web project that can display that functionality…but I wouldn’t need to link them together yet in order to write a blog post about them, or to get the solution to build in a CI system like Travis.

I decided my first effort would look like this:

Write about this plan, and my high-level thoughts on planning solo projects
Create the Trello board to map the whole thing
Write about the Trello board and my planning process
Setup a simple code library I could test in a CI system
Setup a simple web project just to have it ready
Write about the code skeleton

My second effort will focus on:

Setup Travis-CI to build and test my solution
Write about setting up Travis
Write some tests against the web project’s UI
Get them to run in Travis
Write a post about UI tests
Wire the web project to the code library
Update UI tests to reflect the dynamic logic
Write about wiring up the two

After that, I’m not sure exactly. Logically, deploying the program is the next step, but I’m not going to think that far ahead. Sticking with the road trip mentality, I know where I’m going tonight, I know roughly where I’m going to be the night after that…anything further isn’t super useful to plot in detail. I just don’t know what will happen.

Honestly, I’m already second guessing how big that second effort looks. I’m a big fan of bite size pieces. But, since I don’t have a better idea and all of that is grouped logically together, I’m just going to roll with it until a better idea presents itself.

I’ll repeat this process till I get all the backlog cleared. If something needs to get added to the backlog, or needs to go back to the backlog, I’ll do it. If I learn after the next effort that 3 posts is way too much to try and fit into a week, I’ll trim the next one to account for it.

It feels pretty Agile to me, but as a millennial I hate to put labels on things, y’know…

July 28, 2019 0

Q: How Do You Walk From Portland, OR to Portland, ME?

A: One step at a time.

One of the major challenges I had (and still have, if we’re being honest) is looking at a project I want to do and actually delivering it as a solo dev. As time has gone on, I’ve learned that’s not exactly a problem exclusive to solo devs. One of the things that I have really appreciate at Pillar/Industry X.0 is how we approach this problem…and some of it really translates well to hobby code and side projects.

There are two key points I want to harp on:

Know where you’re going (what your desired end result is)
Do only the next thing (focus on what makes sense at the time)

Know Where You’re Going

My personal problem with a lot of side projects is I get an idea, and I start writing code for it. I don’t think about what the end result is going to be, usually telling myself “Don’t get too far ahead of yourself, just flesh it out a bit and it will become clear as you go along.” While I think this is a pretty decent way to write a novel, I’ve come to realize it’s a terrible way to write software.

There is a not-so-subtle difference between planning too much and planning too little. There’s no real benefit in planning out every single thing in advance, from the size and color of your standard buttons to how to promote your new app. But the general idea–web app or mobile? Is this going to live in Azure or AWS or Heroku or what? Am I using GitHub or BitBucket? Should I be continuously deploying this?–helps guide the architecture, the tech stack, and helps define the tasks.

Writing the code is important (sort of the point), and is broken up in it’s own way. But when does it make sense to setup a build pipeline? The repository? Do I need a production environment? When can I actually see changes when I deploy them, anyway? By keeping these questions in mind, by seeing a bigger picture than just features in the code, it helps guide the overall workflow.

Do Only the Next Thing

Having those tasks ranked and in rough chronological order helps you stay focused. Now you can say, “Self, we are going to test drive this feature here. And when it’s done? We’ll setup a build pipeline so it’ll merge into master on it’s own.” Once that pipeline is working, “Self, now we’ll add this other feature here, because when it’s done we can add it to a UI project!”

By modularizing the tasks, you’ll avoid the trap I always seemed to fall into–“I have a web app with no logic behind it and deploying it is a nightmare” or “This code library I spent a month on is useless till I get it plugged into something and I am just too tired to figure out what to do now.” Being able to focus on checking things off a list is satisfying and keeps things moving, especially when you’re navigating a full time day job.

You know you need to cross the country to get from one Portland to the other, but if you focus on that you’ll never make it. You need to focus on getting to the next stop, rather than the whole trip.

The Great Pipeline Post Project

I had this great idea to blog about setting up a CI/CD pipeline, both for much-needed practice and to relay some lessons learned. Spoiler alert: this is a much bigger deal than it seems at first glance. I sat down to get started and quickly got overwhelmed by the sheer volume of things to work through, especially as they occurred to me randomly, while I was sitting in front of Visual Studio.

That’s where this post came from. What if, instead of just breaking down how to setup a pipeline and all the steps under that umbrella, I started with how I planned this mini project? If my target audience would benefit from seeing all the pieces of CI/CD tied together it’s likely they’d benefit from seeing the pre-work too.

Over the next few posts we’ll start by exploring the planning that goes into standing up a new project and move on into the actual work needed. My goal, the whole time, will be to keep it on track and demonstrate how the joint efforts of knowing the plan and doing on step at a time keep things moving at a steady pace.

July 21, 2019 1

Configuration is the Real Magic

A white haired white bearded man holding a wand in the background of fighting brown bears being conjured out of a field of multicolored leaves

I talk about writing software being like magic. It’s how I view the craft in a lot of ways–using arcane words and phrases combined with strange tools to conjure amazing things out of a series of 1’s and 0’s on a circuit board. But when I describe my ultimate career goal as “being a wizard” I mean something, or someone, very specific.

I’m not talking David Copperfield or Penn and Teller. They are very skilled magicians, not wizards. That sort of magic is based on memorizing an illusion, of doing a thing in a certain way because of the reaction you know you’ll elicit. One does a very specific thing, or specific set of things, in a well organized way, over and over and while it gets more polished with practice, it will always be the same.

The words “programming is magic” conjure up Gandalf for me, from the Lord of the Rings universe. For the majority of those adventures, it wasn’t actually magic that made Gandalf the go-to guy–it was never “Gandalf has a spell for that.” What made Gandalf the truly formidable foe and an invaluable ally was the fact he knew things. He understood the old lore, the behavior of people and animals, how the elements of the world fit together and how they could either help or hinder a particular goal.

Gandalf knew the configuration of Middle Earth. Maybe not the *exact* configuration for every situation, but he knew what was missing, or what was the blocker. His magic was always a tool to divine knowledge or to force something into place, to adjust the configuration of a situation. Never magic for the sake of magic, happening in isolation.

“Gandalf knew the configuration of Middle Earth.”

Just like the veteran programmer on the team who always seems able to spot the problem

Continue reading “Configuration is the Real Magic” →

May 19, 2019 0