IBM z/OS Connect / Validating the direction of the product
Validating the direction of the product
Overview
As described in Defining the future direction of IBM z/OS Connect, containerising the z/OS Connect deployment was seen to satisfy many of our customers requirements while also aligning with the corporate direction of IBM.
For many reasons containers felt like the perfect fit:
For many reasons containers felt like the perfect fit:
- It simplified resource management through technologies such as Kubernetes.
- It offered developers isolated environments for development and testing.
- It aligned with the rest of the industry thereby requiring less bespoke training.
- It standardised the deployment artefacts thereby simplifying the creation of DevOps pipelines.
- It enabled z/OS Connect to appeal to a new market interested in running z/OS Connect APIs in cloud environments such as AWS and Azure.
The risk however was alienating our existing customer base who had already invested time and money developing infrastructure to run z/OS Connect APIs natively on z/OS. To validate our direction we conducted a series of user research call, and released a series of betas to understand our users confidence in moving towards container technology. The intentions was to fail fast and pivot if necessary.
User research
During these early research calls we dove deeper into the details of our to-is scenario. We asked our sponsor users to critique our designs and answer the fundamental question: Will this work in your organisation?. In this presentation we took our users through the process of downloading the tooling, running the tooling, then deploying an API into a container environment.
Sponsor users
Sponsor Users are real-world users that regularly contribute their domain expertise to your team, helping you stay in touch with users’ real-world needs throughout the project.
Number of participants
7
Key findings
- Customer were positive and excited about the prospects of a web interface.
- Customers saw no issue with the process of downloading the tooling (as laid out in the to-be scenario)
- Containers were seen to be the right move, however there was hesitance about using them as they were new to the mainframe.
Quotes
I think this is a very good direction - gives much more flexibility for the developers - much easier to set up and configure and enable the developers to get up and running - looks very promising.
System programmer, European bank
This is the right move - however for us on the mainframe and containers does not exist - will take us a while.
Application developer, European bank
We're dealing with x86 (intel machines) architecture, and S390x (mainframe). Need containers that can be run on both architectures
DevOps Engineer, American bank
This research left us feeling confident about our direction. All sponsor users were advocating for the change and asking very poignant questions around the technology stack. This research however only represented a small proportion of our audience and we required more data from the broader audience.
Design Partnership: Containers presentation
Design Partnership
The Design Partnership was a monthly call to 50 of our existing z/OS Connect customers. On these calls we’d present future designs and updates then ask our customers to provide feedback via polls or verbally. It was an infinite source of great feedback for the product.
The objective of this call was to gather feedback from a broader audience about the move towards a containers. On this call we presented our to-be scenario, then polled the audience on a series of questions.
Key findings
- A number of customers suggested that containers were the future direction for their organisation.
- When asked if their organisation used Kubernetes today, 58% suggested they had Kubernetes on IBM Z, on-premise, or on the cloud.
- When asked when they’d be ready to adopt a container runtime for z/OS Connect:
- 40% of the audience suggested they’d be ready within 12 months
- 26% more than a year
- 35% were still unsure about the value of containers
- There was still hesitance from a percentage of our audience who were unsure about moving to containers.
This call proved very promising. We has a large percentage of the audience who had access to a container runtime, and upwards of 60% who were able to adopt containers in the coming years. Nevertheless we still had hesitance from a portion of our existing customers and we needed to understand how this could be addressed.
Beta 1
While the research so far appeared promising, there was a concern that our users lacked practical understanding of containers. This led to release our first beta which was very simple test. We provided our sponsor users with a z/OS Connect server packed as a container image and asked them to run it in a container environment - that simple.
Number of participants
4
Key findings
- All customers were successful in running the z/OS Connect server image.
- While customers understood the basic principles of containers, this was the first time any of them had used the technology.
- 2 customers sought to run existing APIs in the server (which was not yet possible) showing their excitement and anticipation for the technology.
While it was a simple test, the results gave weight to the customers willingness and excitement to use containers. The results however were limited to a small number of users in a small number of companies.
Beta 2
The subsequent beta was more complicated. We asked users to install a beta of Kubernetes on z/OS, then run a z/OS Connect API which called the Db2 database.
Number of participants
2
Key findings
- No customer successfully installed Kubernetes on z/OS due to the complexity of the beta and the time required to install it.
- Users required more thorough documentation which covered a broader set of edge cases to complete the task.
- Neither customer got far enough to ran the z/OS Connect API
While this beta yielded little meaningful feedback, it highlighted that one of our dependencies (Kubernetes on z/OS) was still a long way from completion. As a result we began to explore alternative container runtimes on Z. It also highlighted that the ecosystem around containers can be difficult to set up and will require a large investment in time.
Beta 3 / Openbeta 1
The next beta focussed on our tooling. We provided an amd64 image to our users which could be run on a container runtime such as Docker on a developers workstation. This beta features thorough documentation of the installation process, and further documentation on how to build an API. This beta was initially provided to a sample of sponsor users, then released as an Open beta for public consumption.
Number of participants
X Customer
X IBM stakeholders
X IBM stakeholders
Key findings
Runtime:
- All users were successful in running the tooling.
- Z Developers were unfamiliar to container runtimes such as Docker.
- It was difficult and slow process for Z Developers to obtain a container runtime on their machine.
- Customers would have preferred a free container runtime to be documented in the z/OS Connect documentation (Docker had recently upgraded their licensing so it was no longer a free offering).
- Some customers wanted to run the tooling on Z hardware rather than their local machine.
- Some customers had to scan images to ensure nothing malicious was inside. This took a long time for customers to complete.
Tooling:
- All users were successful in building the sample API.
- Some major bugs were identified (and quickly fixed).
- Users enjoyed the thorough documentation but suggested video content might be easier to follow.
- Customers like the web based tooling and found the interface simple to navigate.
- Customers were blown away by the power of the mapping technology (JSONata), but suggested it needed thorough documentation.
- Customers liked the contract first process.
- IBM stakeholders questioned if all customers would be ready for the contract first process. They suggested more documentation on how to create an OpenAPI document.
- Customers and IBM stakeholders liked the testing capabilities in the tooling.
While uptake was initially slow we gained some very valuable feedback from this beta. With hands on knowledge our customers started to better understand our solution and provide far more meaningful feedback. A key discovery was how difficultly it could be for our customers to obtain a container runtime. Large customers were typically faced with a lot of paperwork which could take months to complete. Nevertheless, once customers overcame these hurdles they were thrilled by the potential of the offering and were eager to understand the next stage of of the process - deploying an API into a test environment.
Design Partnership: Open beta overview
In this presentation to the Design partnership audience we presented an overview of what was included in the Open beta. The objective was to encourage more customers to try out the beta and get more hands on knowledge of the solution.
Key findings
- Customer were excited by the potential of the tool and it’s new capabilities.
- Customers were struggling to access a container runtime capable of running the beta.
- A poll results suggested that 33% of participants did not have a container runtime that could run the beta.
- Customers who had not tried the beta grew more apprehensive.
- A selection of customers had growing concerns over the requirement for a container runtime to deploy the APIs.
- A selection of customers began asking more serious question about adoption:
- How will I adopt this solution?
- How long will it take to set up a container infrastructure?
- What will happen to my existing APIs running natively on z/OS?
- What platform is most performant?
Findings from this call raised alarm bells. Users were struggling to access a container runtime and started to grow more apprehensive. In light of these concerns the team drew up a research plan to better understand the limitations our users were facing.
Adoption research
The adoption research was undertaken in response to our users growing concerns that they’d struggle to adopt the new solution. This research targeted a selection of customers who had raised concerns on the Design Partnership call. These users were asked a series of questions which gave contexts to their concerns, and sought to identify a suitable path forward for the product.
Number of participants
9
Key findings
- There was a lack of knowledge about containers, performance, and how to manage them.
- Customers we unsure of the benefits of containers over the existing server deployment (on a z/OS address space)
- Customers understood that Z software needed to align with other platforms but needed more time invest in the infrastructure.
- Customers were concerned by the time and cost to implement containers and Kubernetes.
- Customers desperately wanted the new features of the z/OS Connect tooling. They couldn’t afford to wait until they’d set up the container infrastructure.
The findings of this research suggested that some customers simply weren’t ready to adopt containers in the short term. They needed more time to set up the infrastructure and better understand the benefits of containers.
Design partnership poll
In a later Design partnership call the audience were asked where they’d prefer to run the z/OS Connect APIs. 76% of the audience suggested they prefer to run z/OS Connect APIs on a z/OS Address Space (like the existing server).
The pivot
Pivot!
In light of this feedback the team had to reconsider its direction. While our customers agreed that containers were the future direction, they simply weren’t ready yet. They needed more time to prepare the infrastructure. Nevertheless they urgently needed the benefits delivered by the new tooling in the short term. To benefit these customers it was decided that we’d deliver a ‘native’ server that could be run on a z/OS address space (like the existing server), as well as the containerised server offering managed by Kubernetes.
General availability
The new version of z/OS Connect was released to the public in May 2022. Uptake of the solution has been gradual as existing customers transition from the previous product version and the sales team seek new opportunities. The team are continuing to measure and understand how the product could be improved. A key aspect of our success has been educating our sellers about the product changes. Details of the sales collateral have been detailed in the following: ’Sales collateral for z/OS Connect’.
Continue reading
