LOGIN
START FREE TRIAL

LogSense Blog

See everything, even before it happens.

PODCAST - Inside Perspectives on Observability, Application Performance Management, and More

Jul 15, 2019 11:32:03 AM |     LogSense Comms
LogSense-PODCAST

We sat down with LogSense founders (Michael Fishman, Przemek Maciolek, Brandon Virgin) and Barracuda Networks CTO Fleming Shi to discuss their perspectives on all things observability (including metrics, traces and logs), application performance management, and other industry trends.

You can read the highlights below or listen to the podcast here.

 

PODCAST HIGHLIGHTS

Moderator:

I'm here today with the founding team and architects LogSense, along with a very special guest Fleming Shi, the chief technology officer for Barracuda Networks. Let’s do a quick introduction of our speakers and share some of the inspiration behind LogSense.

Michael Fishman, LogSense CEO:

We started LogSense because we saw a great opportunity to deliver a more holistic approach to understanding what’s happening in the dev cycle – from both a monitoring and troubleshooting standpoint. Many people were focusing on individual components and trying to piece together solutions. LogSense brings together a very powerful backend, with very powerful collection capabilities and machine learning, to make it much easier for users to figure out what's going on with the applications they're building and the environments in which they're running. From a monitoring standpoint and a troubleshooting standpoint, LogSense puts all the information at their fingertips in a very easy to use UI that makes it a very clear as to how they can solve the problems that their customers are experiencing.

Brandon Virgin, LogSense CTO:

Throughout my career I had a lot of problems that were very hard to solve. Typically what happened is that you'd have network engineers and application developers and DevOps arguing back and forth about who caused the problem, finger pointing to each other, and they were just going back and forth. I always wanted a data driven discussion, something that was based on facts. And this is exactly what we’re trying to solve at LogSense.

Przemek Maciolek, LogSense VP R&D:

I’ve worked with countless organizations on collecting data and extracting value out of data. What I found interesting is that many of the organizations have this data available, but do not have the time or resources to connect the dots across that data. This is very true for modern environments built around with microservices where there's plenty of data available but it's still very hard to solve the problems. At LogSense, we’re giving teams really great observability into what the infrastructure is doing using some special ways of handling the data using automatic pattern discovery and tracing.

Fleming Shi, Barracuda Networks CTO:

I run the advanced technology group at Barracuda Networks. My team looks at technology advancements and how those technologies can help shape our future products. So we look at different tools and different services that are available to our engineers and make sure they're helpful to the point of building our products but also operating the product.

Moderator:

What types of issues were you having or continue to have at Barracuda that led you to LogSense and taking a closer look at its capabilities.

Fleming Shi, Barracuda Networks CTO:

Starting from the beginning, we always had a target to figure out how to increase or improve or maintain our performance with our applications. We have a lot of SaaS solutions out there. By the time we started to build more microservices and the growth of these microservices means that I have thousands – and someday, maybe tens of thousands – of running instances all over the cloud. So just the sheer volume of data that I will be exposed to is hard to manage and then also the potential cost associated with that amount of data. Cost is directly related to how intelligent the product is. If the product is more intelligent, you don’t have to spend as much time – so if you don’t have to waste a lot of time and resources to identify a piece of information, there’s a big cost savings there. If I'm able to make sense of the data, faster, more efficiently – we will be in a much better place. Ultimately, we needed to find a tool that is a far more intelligent, much more builder friendly – and that’s why we started looking at LogSense.

Moderator:

There's a new buzz word in the industry that's gaining a lot of popularity –OBSERVABILITY. How does observability of your stacks, your services, your apps affect what you do at Barracuda?

Fleming Shi, Barracuda Networks CTO:

What we are doing at Barracuda, obviously we have transitioned and are transforming how we actually build software. A lot of the software we build is based on a microservices architecture. This architecture allows you to have a large fleet of services that runs in public cloud or hybrid cloud environments – these microservices then service our installed base, and we have hundreds of thousands of customers running a physical or virtual appliances as well as SaaS solutions that we offer. So as these microservices increase, so does the volume of data that you have to pay attention to that is related to performance. Some of them are related to scalability because we need to always ensure they are running at the top-notch speed and everywhere in the world because our customers are global. Based on that, it's important to actually have that visibility and observability of your application.

Microservices have been adopted by many, many development teams out there – building software and servicing customers, as well as building systems that talk to each other. The industry trend is really driving towards more dispersed workloads that have to be really well connected, high performing, and we have to be able to really see what's going on and be able to identify bottlenecks, or performance issues, or security problems.

Moderator:

Przemek, as with most product capabilities or benefits as they're talked about, as it relates to observability, there's likely more there than meets the eye -- and sorry, no pun intended, but can you break it down in a little bit more detail for me?

Przemek Maciolek, LogSense VP R&D:

Observability is so important for organizations. Observability is built on three pillars:

1 – Metrics. Metrics lets you know about the performance of the computer system.

2 – Traces. Traces can tell you if the system is behaving good or badly, or if there are any issues happening. Traces allow you to get into the details of particular requests. You can find which request has had issues and where, where the bottlenecks are, and so forth. This is great. In fact, many organizations or many companies that are providing APM (application performance management) solutions are stopping there.

But we go farther.

3 – Logs. With LogSense, we give complete observability because we also include the logs, which are the third pillar of observability. Logs allow you to understand what has happened and also why something happened. And this is great for development teams and DevOps teams to understand what caused the issues and how to fix them efficiently, which ultimately makes them run much faster. And organizations are getting a lot of benefit from that.

Moderator:

Can you touch on that second pillar of observability – tracing – and how it relates to discovering bottlenecks?

Przemek Maciolek, LogSense VP R&D:

Tracing is great and allows us to get very precise information about what was happening with the request. So say, if there were services that had separate requests coming in: maybe there was a message queue on the way and some other services, and sometimes those typologies are very complex. This allows us to find the bottlenecks. Even when a bottleneck is found, the cause of the bottleneck is not always clear. This is when logs come in handy.

Legacy tracing or APM solutions assume that the developer will write the custom logs using their API. Custom APIs are very inefficient because you need developers to spend time on changing the applications to work better with tracing. In fact, normally they are already using some logging solutions. For example, in Java they might be using log4j or logback. In Python, there’s Python logging. With LogSense, our implementation of tracing is able to combine information about the trace with the logs.

So each log that is being admitted within a single trace, it's including information about which trace this was, which means you are no longer dealing with tens of thousands of logs for a given trace and you are dealing only with those specific logs that are related to that particular trace. And this approach allows you to find the root cause of the problem very quickly.

Moderator:

Let's shift gears just a little bit and take a closer look at how the system handles unstructured data. What seems to be one of the key differentiators?

Przemek Maciolek, LogSense VP R&D:

Pretty much every computer system generates logs. And the way logs were initially conceived is that logs are human-readable. So there's an operator who can log into the system and check what kind of logs are there. While this is great when someone can have time to go through all those logs, it's very inefficient when you have dozens of services running. So organizations are employing centralized logging systems that collect all the data and make it available to go through them. However, because those logs are human-readable, they are not very machine-friendly.

For example, if you would like to build a chart of certain logs, you cannot do it right away. You need to do parsing first, and parsing requires stamina. You need to understand how to work with regular expressions or use a tool such as Grog. Eventually nobody really does it and there's this huge long tail of logs generated by customer applications that are not being analyzed.

So we are filling this gap with our automated pattern discovery system in the LogSense platform. We convert unstructured data into structured data, which means that right away, without any sort of human input, you can get value out of the data, out of the logs. You can build charts easily on any parameter. With a single click of the mouse, we can employ machine learning. And this is quite exciting because now when the data is structured and parameters for logs can apply some meaning, deep learning is able to extract what's changed and provide meaningful anomalies to the users.

Moderator:

I want to take a step back at this point. Everything we've talked about so far has been the result of data that's been analyzed by the system. Tell me about how data actually gets into the LogSense system.

Przemek Maciolek, LogSense VP R&D:

To get data into the LogSense system, we have many of options available, whatever suits you the best. There are several ways to send a log. Say, you’re an application developer. You can send logs directly from your Java application (for example) that you can integrate with via our logbook integration. For Java developers, it’s a single entry into your maven with the dependency and then a single file that includes some information about what’s being used. With Python, you also can add a single dependency and send the logs right away. The same is true for many other programming languages.

Another way is through log ingestion. Say you already have some log ingestion in place – maybe using Fluentd or Syslog. Again, it's just a single or two lines of configuration to send data out to LogSense. You might also use our API that pushes logs. You can use some of the prepared templates for Kubernetes. So if you have Kubernetes, it's literally only two comments required to send the logs.

In the case of tracing, it's quite similar. For Java, you just need to add the Java Agent with the tracer that instruments the code that allows you to send the data. The same is true for the Python, you add a dependency and for the languages it's the same story. LogSense makes it as simple as possible. I don't think it can be made simpler actually.

Moderator:

Makes sense. Also, Brandon, walk me quickly through the process of how to get started as a new customer to the platform.

Brandon Virgin, LogSense CTO:

To get started with LogSense is really easy. You just go to our website at logsense.com, or you can go to AWS Marketplace. From that point you, can start a free trial, with no cost to get started. It's super simple, and we walk you through exactly how to send your data to us depending on the data you're sending to us, which consoles make the most sense for you. We make it really easy to get started, with no cost, and I believe users will find a lot of value in it.

Moderator:

So then my trial proves itself out. What is the process of moving from a trial customer to an actual production customer?

Brandon Virgin, LogSense CTO:

The process of converting from a trial user to a production user where you're paying money, is super simple. You just provide us a means of payment and convert the account. Your data stays where it's at, and you don't have to do any additional work. It automatically converts.

Moderator:

Brandon, could you comment on the low cost of entry for LogSense? It's probably one of the most cost-efficient solutions out there, and I see that on the website comparison page. Is there a chance I might be missing some features or functionality that I would be seeing in other products at a much higher price point?

Brandon Virgin, LogSense CTO:

LogSense is extremely strong, and it has many features that a lot of our competitors don't have. The reason why we're able to save a little cost for our users is because we spend a lot of time on working for efficiencies in the back end. We pride ourselves on that, and it's something we really focus on and that is reflected in the product.

Moderator:

Fleming, any final thoughts on how the industry is thinking and changing as a result of the way you're expected to build and secure products today? And also, a little bit about the role of tools like LogSense and how they play into your development cycle.

Fleming Shi, Barracuda Networks CTO:

Performance and security are very important and I see the responsibility moving towards the left. What I mean is that it's coming out of just pure operators and IT infrastructure or security folks, and it's becoming more of a responsibility for the developers where the origin of the software has been built. And I think that's why I call it shifting left. Going towards the origin. But what I think is cool about that is that developers need to develop and be more aware about how they're building the software. They need to have probes in place so the performance can be really well tracked and traced, especially in a very large operating environment. If you have thousands of nodes, you need to identify why that particular synchronization call for your database or a particular call to your cache cluster is choking. Right?

And a lot of times it requires a level of depth into the information you need. It's not just purely coming out of logs. It has to be running next to the application. And I think that that's really important to recognize. The same thing for security. Security is another area where folks need to pay attention as they develop the software. From my perspective, the user experience is one of the first impressions you make on your customers. The user experience and performance is a major part of that user experience. So I'm excited to find the right tool to actually give you that visibility and observability.

Moderator:

Michael, you began this conversation, so with this one last question. With so many products out there that sound like they do something similar. Why LogSense

Michael Fishman, LogSense CEO:

There are quite a few tools and it is somewhat of a crowded space on the legacy log management space. But our biggest differentiation for LogSense are the components that Przemek mentioned. Our ability to take unstructured data and make it structured very quickly is unique. So from the moment we are receiving logs, we can go all the way from the receipt of the data to the identification of anomalies – without a human touching it, making it very easy for them to see in the UI what's going on very quickly. And nobody else is doing that. So it's a big differentiation.

 

Moderator:

Thank you. Michael. Brandon, Przemek, and Fleming. We covered quite a bit. If you have additional questions, want to start a free trial, or otherwise, visit logsense.com. At this point, the conversation's a wrap. Have a great day, everyone.

 

New call-to-action

 

 

Topics: Observability

Want more of the LogSense Blog? You got it.
Subscribe to our newsletter.

Comments