There are many services that let you send your logs and search through them. What makes LogSense unique is how we treat your logs, traces or metrics. Whether your data is structured or if its unstructured data - there is no difference on the LogSense platform. You don’t need to make it structured to get meaningful and actionable insights - LogSense does the work for you.
Monitoring Kubernetes Cluster Logs
Let’s take a look at the logs that come from the Kubernetes cluster with the help of the LogSense Kubernetes DeamonSet. They come in a structured way where the container metadata are put in the JSON field and your raw logs are put as one of those fields. When you push the logs into LogSense, they can be filtered by namespace, container name, image, host or many other ways without any action on your side. Those labels can be used together, offering more advanced filtering possibilities.
Let’s take a look at sample logs from the Kafka containers:
- [Times: user=0.00 sys=0.00, real=0.00 secs]
- [Ext Root Scanning (ms): 2.3]
- 2019-03-22T11:40:33.786+0000: [GC cleanup 163M->162M(257M), 0.0010503 secs]
- [GC Worker End (ms): 332651161.5]
When you view these logs, they appear as unstructured strings and each one is different. If we want to take the interesting information from them, we need to create a schema. This is not an easy task as it takes time and every time a new type of log appears, we need to create a new pattern for it again. This is not an efficient use of time but traditional log monitoring solutions require this extra work in order to make the information useful quickly.
Why LogSense and Kubernetes
LogSense is a unique solution because it does the heavy lifting for you. Our patent pending log parsing technology makes it simple to turn radically unstructured data into something actionable and useful - super fast.
We run our pattern matching algorithms and look through your data to help you find variables. For example, let’s take a look at patterns found for the Kafka containers:
With LogSense, you quickly can turn those patterns into rich graphs to better understand how this data changes over time and how those changes relate to other parts of your system - whether it’s raw metrics, traces or logs. Unlike other log management solutions that typically treat this type of data as plain text, LogSense gives you the ability to view this information all in one or many dashboards, depending on your needs. You can sum your variables, or get min, max or average value or percentiles if that’s interesting for you.
Troubleshooting and Forensics
Sometimes there are situations where it’s useful to take a look at the graphs we didn't know would be interesting before. This might be the case, for example, if some production server crashed. Often we might have the data, but we didn’t mark them and cannot create graphs based on the historical information, which leaves a significant gap in the overall story of what occurred to make the production server crash at all.
With LogSense, we find those patterns for you. If you find a pattern interesting, you can use it to look at data from the past. And, with the LogSense auto discovery, as soon as we automatically discover an interesting pattern, we keep track of it so that it’s ready when you are. Overall, LogSense makes investigating problems much faster and easier, so that you can focus on keeping everything running smoothly.
This approach is especially true for the Kubernetes environments, where we run all sorts of applications and all of them are different. Having one place where you can review your logs in a structured way — even though your applications don’t send structured data — is really important. It makes you better and faster to understand what is going on with your cluster. And this is true, even if you run across something you didn’t realize would be an issue later.
With LogSense automatic pattern discovery and parameterization, your system is constantly in a learning and improvement state so that your data is always accessible, always relevant, and always useful. Our patent-pending Machine Learning capabilities identify anomalies automatically - and automatically alert you of changes or issues. For example, if we found that some variables had a low value all the time but suddenly changed - we create an alert that is sent via Slack, or whatever method you prefer. This way we can find potential problems with your applications or cluster before they become a major issue, and you can focus on what matters most - your software.
LogSense is a great companion for your Kubernetes cluster. Whether it’s logs, traces or metrics, we help you get meaningful information from them. Curious how you can get more from your Kubernetes cluster? We would love to show you how LogSense can help.