In a previous post I looked at how the adoption of not only cloud-based software platforms but ancillary cloud-based services has really taken off, supported by a robust ecosystem of third-party SAAS tools for project management, testing, QA, monitoring, analytics, collaboration, etc. In the past it was common for software teams to develop some of these services, such as testing and monitoring, in-house. Others services, such as project management tools, might have been purchased on physical media (first floppies, then CDs, later DVDs) and laboriously installed on individual computers — often in far greater numbers than officially licensed).
Still other services, such as distributed collaboration tools, simply didn't exist before.
In today's cloud ecosystem, these readily-accessible tools are available in ever-increasing numbers. If you know what you're looking for, you can Google something like "agile project management software" to find product information and reviews. But sometimes it's nice to hear about what's new or popular a little more serendipitously.
CTO Favorites and Contenders
During previous research I conducted a survey of fellow CTOs to quantify their use of cloud services. As a byproduct of that exercise I received several CTO suggestions for favorite service which I’ve shared here,
along with notable category competitors also worth considering:
Distributed Agile Standups
User Session Playback
Visual Regression Testing
…Which are your favorites?
Also published on CTO Craft
QA is too-often treated like the ugly stepchild of software engineering, particularly in fast-paced startup-type environments. Look at many teams’ code and you’ll find a hodgepodge of scripted tests, possibly automated (possibly not), with varying levels of coverage and maintenance, driven by individual initiative more than universal mandate.
In this age of agile Continuous Integration (CI), automated tests have become generally accepted best practice, yet in my experience they remain far from a generally practiced best practice.
QA outsourcing has become increasingly popular. It’s easier and cheaper than ever to contract with a human(s) to manually test your software or even use a “crowdtesting” service to mimic automated software tests. Whatever its merits, I believe QA outsourcing dampens adoption of automated testing by making it economically feasible to “solve” the QA problem by throwing people at it. This helps perpetuate software developers’ “we’re too busy to write tests” argument.
Common Types of Software Testing
Before I go further, let’s take a moment to review some definitions:
Now that we’ve gotten that out of the way — what’s your team’s adoption of automated testing?
A. We’ve automated our unit tests
B. We’ve automated our integration tests
C. We’ve automated our functional tests
D. Yes to A, B, and C
E. We have people who test — they’re called “QA”
(If you answered D, congratulations! You can stop reading now.)
Not too long ago I was managing a team whose answer was ‘E’ — we had people for that (in this case, an outsourced contractor). This was a capable, high-performing software development team, quick to adopt new technologies and generally moving fast (and breaking things along the way). The team was adding unit tests, but slowly. They didn’t see writing and automating tests as particularly high priority, especially with a contractor doing a seemingly good job of finding a few new bugs nightly after every code drop. It wasn't broke so they weren't going to fix it.
Perhaps this sounds familiar.
To make the case to invest in developing a robust portfolio of automated tests, I created the following deck:
Automated tests are indeed a lot like robots on a factory floor. They don’t replace the people who conceive of the product (formulate test plans), create initial versions of the product (manually run test plans), design and build the robots (write automated tests), or do new product R&D (perform exploratory testing). But for the large part of QA that consists of repetitive testing, robots (automated tests) outperform their human counterparts and should be leveraged for unit, integration, and functional tests across all software environments.
Also published on CTO Craft
If the tale of Rip Van Winkle were rewritten about a software manager, Rip would fall asleep in 2005 and awaken today to find all of his servers that used to run in the closet sold on eBay for pennies on the dollar, and all of his software running in the cloud.
While the rise of AWS and other cloud platforms is well-documented, ol’ Rip would also be surprised by the scope of the ecosystem of third-party services, all cloud-based, for project management, CI/CD, testing, monitoring, etc. Unlike a cloud platform, these services don’t replace your old servers; rather, they replace your own labor to implement these services and features.
For software developers who have come of age since the start of this cloud-based revolution, the proliferation of third-party cloud-based services is a given. In a recent discussion with one such developer, I argued that our team’s 15 engineering cloud-based subscriptions was “a lot.” The developer looked at me like I had just woken up from 2005 with a long beard. These days, then, what’s a typical number of cloud-based software services?
A non-scientific survey
To find out, I conducted a survey of fellow members of a local group of technology leaders — the DC CTO Club. Based on responses from 11 DC-area CTOs, the number of cloud service subscriptions (not including AWS or other platforms) ranged from 2 to 22, with a mean of 10.6.
To put this in perspective, that means a typical software development shop has chosen “buy” ten-plus times when making a build-vs-buy decision on services available in the cloud. Why are cloud-based services chosen repeatedly? I see three reasons:
Buy vs build vs bag?
Is there any reason to build instead of buy a cloud-based service? Sure. Maybe your HIPAA or other security requirements aren’t met by a third-party service. Maybe you have esoteric technical requirements, like binary-encoded messages, that can’t be satisfied by a cloud-based service like Pusher.
You should also not use a new cloud-based service if you don’t actually and actively need it. This bears mentioning when the introductory cost of many cloud services is minimal and your engineers may enjoy exploring new and cool solutions to problems you don’t have. There’s a always a cost beyond the price — every new service is another moving part that incrementally increases the cognitive scope and complexity of your software. Depending on the service, it may also increase your cloud platform costs, edit-compile-test time, or page-load time.
Don't sleep through this revolution, Rip
Cloud services' ease of adoption doesn't mean you don't have to do your homework — you do. But in your due diligence, you may need to recalibrate your view of how many 3rd-party services qualifies as “extravagant” — look at total monthly spend across services, not number of services. For software infrastructure that adds value but is not core to your own development expertise, a robust, competitive ecosystem of third-party cloud vendors means you’re more likely than ever to find your needs met by a new service subscription (even if it’s your 10th).
In another post, I’ll follow up with some of my and others’ favorite third-party cloud-based services.
Also published on CTO Craft