A couple days ago, Steve Smith started an interested conversation on Twitter by pointing out that using a normal CI tool to manage your CD pipeline is a mistake.
I agree with Steve 100%. But in a moment of snark, I took the conversation into a place where we found disagreement:
So, why would I suggest something that seems contrary to our agreed upon position that using CI tools for CD is an anti-pattern? Well, it gets to definitions of things, subtle distinctions and the like. If you're looking for simple lessons, stop here and don't use a build tool for deployments. If you want to understand why I think CI and CD done correctly are almost the same thing, follow me down the rabbit hole. Read More...
Both IBM UrbanCode Deploy and Release have had some nice improvements over the past couple months that I want to highlight.
Fixpack versions 188.8.131.52 and 184.108.40.206 dropped in February and March respectively. Both have a few bug fixes, and an emphasis on performance improvements.
There are a number of new and improved integrations as well. New integrations include deployments to Android, working with Microsoft TFS Work Items, and passing information to UrbanCode Release. Partners also delivered integrations to Appurify, Datical and Shunra. Updated integrations include Websphere (both application deployment and configuration), JBoss, Jenkins, and Worklight. You can get more details on the updates in February and the updates in March.
To stay posted on those updates, you can watch the general “What’s New” page for UCD.
The IBM UrbanCode Release delivered three updates in the last two months. Release 220.127.116.11, 18.104.22.168 and 22.214.171.124.
These updates are fixpacks and so are focused on bug fixes and performance improvements. However there are a handful of very useful small improvements, as well as new “Experimental Features”. Experimental features are ones which are disabled by default, but interested teams can turn them on to provide us feedback before we fully include the capability as part of the tool. I’ll blog more about experimental features later.
126.96.36.199 is definitely the biggest of the three and added the ability to copy a release, request deployments based on quality gates, and hide tasks that aren’t relevant for this deployment. Experimental features include new views for federated deployments; mile stone patterns in lifecycles where milestones are automatically attached to a released a set number of days prior to the target release date; and the ability to reserve an environment for a release.
To stay posted on those updates, you can watch the general “What’s New” page for UCR.
In recent times, a number of organizations have realized the need for deploying their products and services, continuously and actively, into production. The advent of a social-mobile lifestyle coupled with market forces to be at the latest and greatest versions of software is fueling such a developmental and operational mind shift. There is a tremendous amount of pressure on software development and IT functions in organizations for pushing all recent updates, both quickly and accurately, to their systems of engagement and their systems of record. As a result, they are making huge investments into adopting DevOps best practices around continuous integration, continuous deployment & continuous test and are developing what are called delivery pipelines that help them validate changes at multiple, parallel stages of their product release cycles. With this blog post, I would like to showcase one such pipeline that will be created by bringing multiple IBM UrbanCode Deploy environments together.
Michael Brockmann details how his team has helped a client support aggressive growth by delivering software more quickly with less risk. The goal was a continuous delivery pipeline using a full stack approach (“phoenix environments”). This effort hits all my favorite buzzwords: CD, Blue/Green Deployments, monitoring, global reach, etc.
Michael goes into more depth with a series of blog posts:
Cloud is such a big topic, here’s a quick look at various cloud options and how they fit in to your DevOps efforts presented by Dan Berg, IBM’s Chief Architect of DevOps Tools and Strategy.
One of the riskiest and most manual parts of a deployment is updating the database schema. We’ve argued for years that you should automate these processes. The most common first step in that direction is to take the SQL scripts you’ve been using and treat them as incremental versions in IBM UrbanCode Deploy. That works, and the plugins for common databases like Oracle and Microsoft SQL Server make it easier.
We are big believers in capturing a “version” of the database as a whole that you can point at your prior database and it will figure out what changes are missing and make them in the correct order. Long time users of UrbanCode products will know that upgrades don’t come with instructions for the DBA. The installer just figures it out. Some of our customers are using the same utility and so there’s a plugin available for that.
Today, the database deployment problem is becoming increasingly common as organizations are getting the rest of their stack automated. The DBA’s are becoming a big bottleneck. Not surprisingly, there are some new solutions in this space. One that builds on the popular open-source LiquidBase is provided by Datical with a focus on enterprise capabilities and ease of use. There’s some pretty cool stuff in there, and now they’ve introduced a plugin for UrbanCode Deploy.
You should check out their demo of the integration:
Continuous integration is when we constantly validate that our latest code changes work with the rest of the application. As applications increasing use composite architectures (assembled from numerous services and data sources at run time) the interesting integration checks are no long within the a project’s code base. Instead, they require functional testing. At its most simple, that means quickly moving changes into the test lab with deployment automation and running tests. But getting a realistic test environment can be difficult. What if some of the services are unavailable or expense to use? Then you add service virtualization into the mix.
The testing challenge can be more severe though. Many times our test lab is a handful of machines and devices in a single data center while in the field, our composite application is spread across multiple data centers and mobile elements are, well, mobile. Accounting for real world network characteristics can be critical when trying to speed feedback. That’s where network virtualization comes in.
I recently joined Todd DeCapua of Shunra Software for a webinar looking at the challenges of continuous integration for composite applications. Check it out.
Peter Spung (@paspung) has started a great series on DevOps speaking to the middle management layers.
Peter introduces the topic in the first blog post. In the second post, Peter starts to look at the disruptive nature of software in industries as diverse as automotive, consumer goods, financial services and retail. Effectiveness at delivering software is correlated positively with success in the market. In the third post, valuable outcomes from a DevOps transformation are reviewed.
As part of our move to new infrastructure we now have the ability to include plugins for UrbanCode products made by our partners. The first two have been listed on our plugins site, Ibmdw.net/urbancode/plugins/.
Appurify provides an API accessible Cloud™ of real mobile devices with run-time debugging, optimization, and testing tools. Companies can automate testing of their apps on real devices directly from IBM® UrbanCode Deploy. The plugin allows you to provide an application, a testing script, and define the device type and environment characteristics and invoke your tests. Pretty nifty.
Details are on IBM PartnerWorld.
The Appurify team has also posted their source code.
Shunra Network Virtualization
Shunra NV integration for UrbanCode Deploy enables developers, testers and integrators to include network conditions into their testing, so communications between the tested application, virtualized users and virtualized services reflect the characteristics that will be experienced in production. Basically, if you need to see how your application will behave with extra latency between the front and back ends, or how your mobile app will work on one bar of connectivity, Shunra NV helps you test those scenarios.
Details of the integration are on IBM Partnerworld.
Reminder: Continuous Delivery is about Testing
Seeing these integrations helps remind me that DevOps is addicted to feedback, and a continuous deployment pipeline enables rapid testing as much as anything. Testing our complex applications means quickly getting changes into test environments and validating those changes as quickly as possible with as real-world a scenario as possible. With these two integrations, we see hooks into some clever technologies for getting to “real world”. Appurify streamlines testing on a real devices, while Shunra helps us test network conditions that are considerably worse than what we’d find in the test lab.
Building feedback loops by automatically kicking off testing after a deployment is something discussed in our white-paper: Deployment Automation Basics
In my previous blog entry, I discussed how to look at how a process involving corporate finance approvals of a new environment from a DevOps perspective. This time, I’d like to share a story about how Legal and Development found a way to drive innovation.
At a big IT shop I worked with, Legal was concerned that with some OSS licenses, the firm would be at risk if libraries were included in their software – they might even have to reveal proprietary code. Security was also a concern. To allow libraries to be used, a process was put in place where developers could request to use some library, and several weeks later Legal and InfoSec would render a verdict. At that point, the library could enter the code base. The “several weeks later” part didn’t sit well with developers who wanted to experiment and innovate rapidly. The deluge of requests from development caught Legal off-guard and was expensive process.
Cooperatively, an alternative approach was set found out that lowered Legal’s costs, met Development’s velocity concerns, and pleased InfoSec. The Legal team reviewed the most common 10-15 OSS licenses (versioned). Those were either blessed (Apache 2.0) or rejected (GPL) for use in the firm’s software. Developers could immediately begin developing against anything with an approved list so long as they notified Legal (who could sanity check the license version) and InfoSec (who could run their checks / scans). Almost always, everything was fine. A small percentage of the time, the developer was informed of an issue and might have to rework some things.
Like the example of corporate finance, we see that that challenges to rapidly delivering good software can span organizational silos well outside of IT and the lines of business being served. The resolution looks DevOpsy. The concerns of various stakeholders are addressed, and flow of innovation is maximized without sacrificing risk controls.