I recently started spending a likely-unhealthy amount of time watching TikTok videos. This is only of interest (possibly) to blog readers because it led to undertaking this little piece of video art.
That’s right. Over the course of a month or so, I dedicated free time to test driving a non-trivial implementation of FizzBuzz. It involved not just FizzBuzz, but a SongPlayer, a song structure, and a console app to display the process. I also spent way too much time figuring out how long Fizz and Buzz and FizzBuzz should be played, in order to best sync to the music.
If you’re at all interested, I have the whole thing in a Github repo. This should at least explain why I haven’t been continuing the battleship game walking skeleton…
There are a lot of CI/CD systems out there. If you’re in .NET land, Azure DevOps has some pretty excellent tooling out there. Jenkins is another that has some extremely robust capabilities. There are oodles more I’ve never even brushed up against, including rolling your own on a virtual machine somewhere in the cloud (or even on your own machine).
But wait. What the heck is a CI/CD pipeline? Why do you need one? If you’re asking either of those questions, please take a look at this recent post of mine.
It integrates beautifully with GitHub (which is my repo of choice)
It’s platform agnostic (works equally well for a variety of tech stacks)
Step One is to get yourself a Travis account. This is absurdly, almost suspiciously easy: you sign up with your GitHub account.
This means you may need to allow Travis as an authorized connection or app in your GitHub account, because Travis will be accessing your repo in order to detect changes and to clone the repo so it can build the application.
You’ll end up at your dashboard, which eventually will display your most recent build and a quick link to your repositories–for now, you shouldn’t see anything.
Step Two is to link a repository, which means you need to access your profile settings. Click your avatar in the top right of the screen, then the “Settings” link.
Step Three is to find and switch on your repository so Travis knows to check it for builds. Your public repos are all listed under the “Repositories” tab, which should be the default view (private repos required a paid subscription with travis-ci.com, currently).
It can take several minutes for your repositories to show up in Travis the first time. You may have to click the “sync account” button. It also may take refreshing the browser window itself–I once waited almost ten minutes to see a repo list and as soon as I refreshed the browser it all showed up.
Find the one you want to activate, and toggle it on.
Now Travis is going to be scanning that repo through GitHub APIs and looking for a .travis.yml file to get build instructions. Which means…
Step Four is to create a .travis.yml file in the root of your repository. The dot-travis file is, as you might guess from the dot starting the name, a configuration file used by the Travis-CI build system to know what language you’re using, what build environment to use, what commands to run, and any deployment steps. It’s sort of a big deal, and if there’s no .travis.yml file found in the code pushed to GitHub, Travis simply won’t do anything.
Again, you want to create an empty file in the root of your repository–not necessarily the root of your code base. Find the .git folder, and save your yml file there. I do this step typically in VS Code or Notepad++, depending what OS I’m on.
Step Five is to define the build configuration. This will vary significantly depending on which language you’re developing in and what flavor of that tech stack you prefer, and the folks at Travis do a really great job documenting what goes in the .travis file.
The .travis.yml file I will use for this project including some helpful (to me, anyway) comments about what each line means…
# Mono is used to build .NET on Linux--we don't need it with Core.
# Needed to run commands in the Xenial CLI
# The version of Ubuntu to run the Travis virtual machine -- needed for .NET Core
# Your SDK version, not your run-time version
script: # These commands are executed in the Travis VM just like you would on your local machine
# Use a "cd" command to move the Travis command line prompt into your solution directory
# - cd /home/travis/build/<yourTravisAccount>/<yourRepoRootDirectory>/.../<yourSolutionDirectory>/
- cd ./BattleshipTDD/
# Use these to build the project without tests
# - dotnet restore
# - dotnet build
# if your project has tests, you can skip "restore" and "build" and just use "test" command -- "test" will run the other two automatically
- dotnet test
Step Six is to commit the new file and push it up to your repo, where Travis should catch the change and trigger a build.
Step Seven is to review the build in Travis. Even before I was done prepping the terminal photo above for the post, I got an e-mail from Travis saying my build passed–hooray!
When we look at the dashboard we get some key items right away. The repository name and a build badge are at the top, and then the specific build info–what branch was built, the commit message, commit ID, the build number, how long it ran for are all front and center.
If we keep scrolling down, you can see the actual build logs, starting with info on the build environment–handy for debugging problems, like when your build is fine locally, but fails when Travis tries it.
For instance, if one tries to run a .NET Core app in the default Travis Linux OS, it won’t work–hence needing to specify Xenial to make sure we were running on Ubuntu. Being able to compare the build environment with the suggestion on a help document was key to figuring this out.
Scrolling down further we get to the actual build process. Each command you specify in the .travis.yml is listed out separately–you can see on line 246, even the “cd ./BattleshipTDD/” got it’s own result output (and yes, I have seen this fail, especially on complicated solution structures where I was missing a directory level in the path).
The test output is particularly helpful–my one test passes here, but should it fail it outputs the exception message just like any other test runner, letting you know what failed and why.
At the very bottom, the build status gets reported–if any of the commands exited with anything except a 0 code, the entire build is marked as a failure.
Some examples of unhappy build logs…
Failing tests give you the output of your testing framework–no guessing what happened
Wrapping It All Up:
What we’ve done here is linked your GitHub to Travis-CI, which will build the project and run tests against it whenever changes are pushed to master. This by itself may not provide a lot of value (I mean, if you’re not running your tests before pushing code, we need to talk) but it does lay an important foundation.
From here, we’re able to automate deployment, headless browser testing, dockerizing, packaging, updating a badge on your repo to tell the world your code is sound, and more. And it happens without having to remember it–push the code, and Travis checks it. It’s one heck of a safety net, if nothing else. It lets you focus more on the code than on the boring devopsy* stuff.
*I mean, I don’t exactly think devops is boring stuff, but no shame if you do–most programmers get into code to write code, not manage deployments and QA etc etc etc.
So what is CI/CD, what does it do for you, and do you need it?
I did a quick search on the interwebz before deciding to write this and what I found at a glance fell into 2 categories:
People selling a service
People hyping a trend
Since my target audience is, largely, a solo dev who doesn’t really have the context needed to parse either of those kinds of posts, I wanted to break it down a little more succinctly.
But First, Some Vocab:
CI (continuous integration):
What it Is: The act of integrating the code being worked on into the main code base on an on-going basis, rather than waiting until a feature is complete and you’re ready to release it. Typically, merging code or pushing local changes triggers a service (a CI system) that will pull the new code, build it, run unit tests, and let you know if you managed to break your application with the latest changes. Code repos like GitHub can be configured to block pull requests that have a failing CI system build. Often, the CI system will archive the built application so it can be used later in deployment.
Problem it Solves: At root, it helps you limit the damage you can do to your code at one time. Rather than checking out a feature branch, working on it for weeks, then trying to merge it back into master and finding out your feature code is incompatible with the master branch with 231 merge conflicts (or failed tests), you can catch issues almost immediately–when they’re much easier to fix. With multiple people in a code base, this gets even more important; everyone gets aligned far more often and reduces the amount of effort needed to keep things moving in the same direction.
CD (continuous delivery/deployment):
Is it Delivery or Deployment? It depends partly on who you’re talking to (I personally don’t view them as different things and will use them interchangeably) and what the context is. If my code is going to some other department that will do the rest of the dev-opsy stuff, it makes more sense to call it continuous delivery–you’re delivering it to someone. But, if your team is responsible for getting it to the users via a webservice or some other process, then it’s more of a deployment of the software. Context is the key, friends.
What it Is: Either way, this is the process of continually putting finished software where it needs to go. This could be a staging environment, a testing environment, or directly to the users if you’re a company like Facebook (or you’re just trying to show off to family like me). A common approach is the CD system will wait for the CI process to successfully complete, and then once it knows the code is good it will either take the output of the CI process (generally referred to as a “build artifact”) and deploy it based on the configured settings, or it creates a whole new artifact for the deployment.
Problem it Solves: In the not-so-distant past, once software shipped it was done, ready or not. Think about all the games you’ve played with crazy bugs and glitches in them that you just don’t see in modern games–games these days can get patched without inconveniencing player or developer. That’s continuous deployment at work. Finish some code, ship some code. It also lets you put a basic application in front of someone, say a client, to get instant feedback–and then just keep adding functionality.
Is It For You?
So if you’re new enough to all this, is CI/CD something worth setting up? I mean, it depends…but my short answer is yes.
It’s not as arduous to setup as it may sound, especially when you look at guides put together by people trying to sell you things. If you do it as one of the first steps in your process, it’s fairly easy to get done–and it really is a velocity accelerator. You won’t get caught by surprise trying to merge in new features (“When did THOSE tests start failing?”) and don’t have to fumble around moving files in order to make sure things work or to show off a weekend of work.
Of course, there’s also the fact that those are the kind of skills that separate someone who can write code and someone who can delivery value to an employer or client. While writing software is a challenging a rewarding pastime, the best program in the world isn’t really any good until someone can put it in front of users.
I’m assuming that you know the basics on how to create a code base in your language of choice, in your IDE or text editor of choice. This series is aimed at folks who can stand up a console and/or web app on their own, with really high-level examples in C#. The goal of the series is to help you start leveraging CI/CD automation, not learn how to set up your dev environment.
I’m also assuming you’re working with Github. There are plenty of other ways to do git, and you may be able to use your service of choice in some places–but that’s on you, my guide will be based on Github. If you want a run down on this before we get rolling, this is a pretty solid article.
Okay, so why skeleton code? What does that even mean? Let’s get into it.
If your goal is to write some code as a kata, and don’t really intend on deploying it anywhere or configuring a pipeline, you don’t really need to be simple. You can go hog wild, and test drive as much as you want as fast as you’re able.
But if, as this whole exercise states, the goal is to set up a project that can continuously integrate and deploy by automated means, the absolute last thing you want to do is put together a bunch of code with moving parts. The more moving parts to your code, the more things that can go wrong. Add in all the moving parts to configuring a CI/CD pipeline, and you’ll very soon wish you had fewer things to troubleshoot.
Okay, How Simple?
For the “business logic” you should have one small passing test. How that is architected is up to you–I’m doing this in C# and you can eyeball the repo for yourself. The standard convention is a class library project and a unit test project, with one test class for each class in the library project.
You’ll also want a super bare-bones web app. Some languages/frameworks (like ASP.NET Core) build quite a bit into their “default” templates, so chances are pretty good you can get a basic “This is my webpage” type app without having to write any code at all. In my case, I changed the HTML on the index page just to amuse myself.
And that’s it. The webpage should appear when you run it locally, and the tests should pass when you run them. Otherwise, you don’t actually want your code to do anything yet. The less that can go wrong with your code, the easier it will be to setup the pipeline.
Once you have that code set up, with tests passing and a webpage viewable, go ahead and push it up to your repo. Then you’ll be ready for the next step, linking your repo to Travis-CI.
If you’re a professional developer, and your team isn’t using a ticket system, you’re probably familiar with card boards. You may even be a Trello wizard–I personally have spent more time in Microsoft’s Team Foundation (now called Azure DevOps), but it’s the same idea. A way to track what needs to be done and how long it’s taking.
Since this is a (relatively) simple one-off project, I just threw together a Trello board. Here’s the 800-mile-view:
My backlog is the total sum of what I want to get done/cover in my post series. I have code items (labeled in orange), infrastructure items (labeled in green), and posts to write (labeled in yellow). I have them roughly grouped in order by things I want to do for a specific post.
Current Effort is what might be called a sprint–it’s the group of cards I think are obtainable to get done in the next week or two. To continue my trip analogy this is just the next leg of the trip–maybe how much ground I want to cover before stopping for dinner. It would be the point where I would feel comfortable putting in a pull request, a point where if need be I could say “I have to put this aside for three months” and be able to pick it up as a complete module of the larger project. The acceptance criteria for each “effort” is to have one or more blog posts scheduled.
In Prog is just that–in progress. Stuff that I’m still working on, since I anticipate doing little bits throughout the week.
Done should be another self-explanatory item. It’s all cards I’ve completed–not just during one effort, but over the course of the whole project.
My first step was to fill the backlog. Everything I needed. For instance, I knew I needed a simple code library to provide some sort of functionality. I also knew there’d need to be some sort of web project that can display that functionality…but I wouldn’t need to link them together yet in order to write a blog post about them, or to get the solution to build in a CI system like Travis.
I decided my first effort would look like this:
Write about this plan, and my high-level thoughts on planning solo projects
Create the Trello board to map the whole thing
Write about the Trello board and my planning process
Setup a simple code library I could test in a CI system
Setup a simple web project just to have it ready
Write about the code skeleton
My second effort will focus on:
Setup Travis-CI to build and test my solution
Write about setting up Travis
Write some tests against the web project’s UI
Get them to run in Travis
Write a post about UI tests
Wire the web project to the code library
Update UI tests to reflect the dynamic logic
Write about wiring up the two
After that, I’m not sure exactly. Logically, deploying the program is the next step, but I’m not going to think that far ahead. Sticking with the road trip mentality, I know where I’m going tonight, I know roughly where I’m going to be the night after that…anything further isn’t super useful to plot in detail. I just don’t know what will happen.
Honestly, I’m already second guessing how big that second effort looks. I’m a big fan of bite size pieces. But, since I don’t have a better idea and all of that is grouped logically together, I’m just going to roll with it until a better idea presents itself.
I’ll repeat this process till I get all the backlog cleared. If something needs to get added to the backlog, or needs to go back to the backlog, I’ll do it. If I learn after the next effort that 3 posts is way too much to try and fit into a week, I’ll trim the next one to account for it.
It feels pretty Agile to me, but as a millennial I hate to put labels on things, y’know…