Splunk platform upgrade readiness app
summary
User: Splunk administrators and developers
Problem: As Python updates from version 2 to 3, it will cause breaking changes in Splunk app code that admins and developers will need to update, and there currently is no streamlined process for this.
Method: Iterative design approach with sketching, prototyping, and concept validation with users
Solution: A Splunk app that admins and developers can download in order to identify breaking changes and recommended actions in their other apps.
problem
When it comes to enterprise software, everything can feel complicated sometimes. Layer on an industry-wide code update, and you might yank out a few hairs. We knew our users might feel the same way. For Splunk Enterprise, the recent migration from Python 2 to Python 3 was the biggest risk to adoption for our latest release, Enterprise 8.0, due to numerous breaking changes to scripts and apps. To help admins and developers transition more seamlessly, our team developed the Splunk Platform Upgrade Readiness App, which aims to help users* identify all impacted components on their Splunk deployments, and provide guidance on how to improve or fix them.
* In this scenario, our users are admins and developers, but for simplicity I’ll just refer to them as “users” going forward.
We encountered some challenges along the way, seeing as this app was going to be used by almost every customer we had. Two things we had to figure out include:
High information density was inevitable. We had to figure out how to convey a useful amount of information in one place without entirely overwhelming the user.
The app needed to support a high number of edge cases, and we had to make sure our user research approach was dynamic enough to investigate these edge cases.
process
design
In this project, there wasn’t a lot of wiggle room for innovation or ideation. We had clear cut requirements and workflows from product management — this is how the app needed to work: The app is first installed onto a Splunk deployment, and then scans all the other apps on the deployment for scripts that will prevent Enterprise 8.0 from working properly. The scan looks for 8 specific code problems as it runs through the deployment. Once the scan is complete, the app displays a list of results which include scripts that have issues, which problem it corresponds to, and a description of suggested action.
Since we were on a time crunch, the initial design process happened really quickly. The product manager and I rapidly whiteboarded workflow logic and basic UI structure based on initial product requirements, and I translated them into early low fidelity wireframes.
Our primary goal was to display the numerous scan results in a digestible way, which forced us to deeply understand users’ workflows and what information they would leverage to make decisions about how to update their deployments. We could choose to sort and display information in a few ways: by script check, by app, by severity of issue, or manually, but without the opportunity to conduct research first, we had to make some best guesses.
As I further refined the wireframes, I began adding fidelity so that participants in our user research sessions could understand the full extent of detail and information this app was going to provide them.
Validation/User Research
Once a more high fidelity prototype was created, our engineering team was able to build a beta. We asked 4 customers to use the beta and provide us feedback through semi-structured validation interviews, led by myself and the product manager.
We hoped to understand whether the functionality was useful to users, what their upgrade workflow looks like, and most importantly, whether the results were comprehendible and actionable. Our feedback was fairly minor, and I was able to integrate improvements into our prototype pretty quickly.
Development
I worked directly with the front-end engineering team to ensure quality of design implementation. The process was quick and direct, and consisted mainly of back-and-forth conversations, allowing me to provide feedback on their builds, and also allowing them to clarify intended functionality when needed.
Iteration
As this app hit the market, and as the need for Python upgrades grew stronger, we began to field requests for additional features. With these opportunities, I had an opportunity to slightly refactor the original design, while continuously ensuring scalability of the app so that its scope could continue to expand.
The first experience improvement we made was allowing users to select a subset of apps to scan, rather than forcing them to scan every app in their deployment.
The second experience improvement we integrated was the ability to schedule scans on a regular cadence, and also some reorganization to the way results are displayed.
Collect Feedback
At our annual user conference, we were able to speak with users who were particularly interested in the Python 3 migration, which gave us an opportunity to promote our app and answer any questions our customers had, but also encourage them to use our app and provide feedback via a survey.
Conclusion
The final product can be downloaded from Splunkbase today. It currently has 2369 installs and 6189 downloads. The product has received praise from users who appreciate our straightforward messaging, organized information hierarchy, and data accuracy.
During my time on this project, I learned to separate visual design from user-centered design. You can come up with a clean-cut, visually appealing masterpiece of a design solution which looks gorgeous in Figma, but just because it’s striking doesn’t mean it provides the best experience possible for users. More than once I would come up with a flashy UI design that simply wasn’t realistic to implement. In these scenarios, it was important to take a step back, really think about what the user needs, and discard your own biases about how something should look. Having to balance these two design attributes taught me an important lesson on understanding what UX design really is: managing a user’s emotions toward the product while remaining practical, usable, and understandable.