Kubernetes Web UIs in 2019Posted: | More posts about kubernetes
This post takes a look at different open source Kubernetes web UIs, my requirements, and why I created Kubernetes Web View to help with support and troubleshooting across multiple clusters.
At Zalando, we have a large number of Kubernetes users (900+) and clusters (100+). There are two main use cases where I would wish for some web tool support: 1) support interactions with colleagues 2) incident response and investigation.
I observed that support interactions in chat often look like:
- A: Help our service XYZ is not reachable
- B: What do you see when you do kubectl describe ingress ...?
Or similar for CRDs:
- A: I am facing an issue with identity service, ...
- B: What is the output of kubectl describe platformcredentialsset ...?
These interactions follow a pattern of using the kubectl command line to investigate and pinpoint a problem. This requires context switches on both sides: participants have to switch between terminal and web chat, plus they don't see the same situation.
I would wish for a Kubernetes web frontend to help with this:
- users can share links and see the same situation
- avoid human mistakes during support: login to wrong cluster on the command line, typos in CLI commands, etc
- allow crafting custom views to send to colleagues, i.e. add label columns, show multiple resource types on one page
- ideally the web tool should allow deep links into specific YAML sections (e.g. to point to a wrong spec causing the issues)
Incident Response & Investigation
Incident response for infrastructure requires gaining situational awareness, the ability to assess impact, and finding patterns across clusters. Some real-life example situations:
- a critical production service has problems, you need to find the Kubernetes resources by name across all clusters in order to help troubleshoot
- your nodes start failing on scale-up and you need to find all "Pending" Pods across all clusters to assess the impact
- some users report an issue for a DaemonSet which is deployed in all clusters, you need to find whether there is a problem across all clusters
My standard solution for this tasks is something like: for i in $clusters; do kubectl ...; done --- obviously there could be a web tool providing similar functionality.
Existing Kubernetes Web UIs
The open source world of Kubernetes web frontends is not very big , so I tried to gather more input on Twitter:
"K8Dash is the easiest way to manage your Kubernetes cluster."
K8Dash looks pretty nice and feels fast, but has some shortcomings for my use cases:
- Single cluster only
- Sorting and filtering is possible, but has no permalinks
- No support for Custom Resource Definitions (CRDs)
“Kubernator is an alternative Kubernetes UI. In contrast to high-level Kubernetes Dashboard, it provides low-level control and clean view on all objects in a cluster with the ability to create new ones, edit and resolve conflicts. As an entirely client-side app (like kubectl), it doesn’t require any backend except Kubernetes API server itself, and also respects cluster’s access control.”
That's a pretty accurate description for Kubernator. Missing points for the described support & incident response use cases:
- Single cluster only
- No list view (i.e. no way to list all pending pods)
“Kubernetes Dashboard is a general purpose, web-based UI for Kubernetes clusters. It allows users to manage applications running in the cluster and troubleshoot them, as well as manage the cluster itself.”
Kubernetes Dashboard will sadly not really help with my support & incident response use cases:
- No permalinks, e.g. when I filter resources or when I change the sorting.
- No easy way to filter by status, e.g. to see all "Pending" pods.
- Only support for a single cluster.
- No support for CRDs (this is being worked on)
- No custom columns (e.g. label columns like kubectl -L).
Kubernetes Operational View (kube-ops-view)
“Read-only system dashboard for multiple K8s clusters”
Kubernetes Operational View has a completely different approach: it only shows cluster nodes and pods rendered via WebGL and has no textual object details. It's good to get a quick overview of a cluster state ("pods crashing?"), but does not really help much in the described support and incident response use cases.
Kubernetes Resource Report (kube-resource-report)
“Report Kubernetes cluster and pod resource requests vs usage and generate static HTML”
Kubernetes Resource Report generates static HTML reports to report resource usage and cost attribution per team/application across clusters. The report helps to a limited degree also for support and incident response as it allows to quickly find the cluster where an application is deployed to.
“A web-based, highly extensible platform for developers to better understand the complexity of Kubernetes clusters.”
VMWare's Octant is fairly new and I only learned about it recently. It's pretty nice to explore a single cluster from the local machine (even comes with visualization), but it only covers support & incident response to some extend:
- No search across clusters
- To be used on local machine (not deployed to cluster)
- No sorting/filtering of objects (only label selector is supported)
- No custom columns
- No listing of objects across namespaces
I also had problems getting Octant to work with Zalando clusters: it crashed on certain CRDs.
Introducing Kubernetes Web View
"kubectl for the web"
Looking at the UI options, I decided to create a new web frontend for Kubernetes: Kubernetes Web View. I really just want a way of translating the power of kubectl to the web:
- enable all (read-only) operations where people commonly use kubectl as their tool of choice
- all URLs should represent the full view state (permalinks) in order to make them shareable among colleagues and facilitate deep-linking from other tools
- all Kubernetes objects should be supported to be able to troubleshoot any kind of problem
- resource lists should be easily downloadable for further processing (spreadsheet, CLI tools like grep) and storage (e.g. for postmortems)
- selecting resources by label (similar to kubectl get .. -l) should be supported
- composing views of different resource types should be possible (similar to kubectl get all) to provide a common operational picture among colleagues (e.g. during incident response)
- adding custom "smart" deep links to other tools such as monitoring dashboards, logging providers, application registries, etc should be possible to facilitate troubleshooting and incident response
- support multiple clusters to streamline discovery in on-call situations (only one entry URL to remember)
- facilitate ad-hoc analysis where possible (e.g. with download links for resources across clusters/namespaces)
- provide additional deep-linking and highlighting, e.g. to point colleagues to a certain part of a resource spec (line in YAML)
- allow customization for org-specific optimizations: e.g. custom view templates for CRDs, custom table views, custom CSS formatting
- provide means to continue investigation on the command line (e.g. by showing full kubectl command lines to copy)
Out-of-scope (non-goals) for Kubernetes Web View are:
- abstracting Kubernetes objects
- application management (e.g. managing deployments, Helm Charts, etc)
- write operations (this should be done via safe CI/CD tooling and/or GitOps)
- visualization (check out kube-ops-view)
- cost analysis (check out kube-resource-report)
How does Kubernetes Web View help with support and incident response?
- all links are perma-links making it easy to share views with colleagues
- custom views can be crafted, e.g. to show all Deployments and Pods with a certain label in two specific clusters (multiple cluster names and resource types can be separated by comma in the URL path)
- individual lines in the object YAML can be linked to point to potential problems in the object spec
- the global search allows finding objects across all clusters
- list views can show all objects with a certain state/column across all clusters (e.g. to find all pending pods)
- object lists can be downloaded as tab-separated values (TSV) for further analysis
- customization with external links allows fast jumps to relevant monitoring dashboards and other tools
The UX can certainly be improved and Kubernetes Web View is currently a tool for "power users" who don't shy away from manipulating the URL path if necessary ;-)
If you have any feedback or improvement suggestions, please reach out to me on Twitter!
This post should just provide a brief background on the journey leading to Kubernetes Web View. More updates to come!
|||My explanation for the small world of Kubernetes web UIs: cloud providers and Kubernetes vendors have their own proprietary web frontends and therefore there is only a limited space for "good" open source Kubernetes UIs.|