ChaosSearch has experienced tremendous growth as evidenced by being named one of DBTA’s Trend-setting products for 2022 and one of three vendors chosen by Solutions Review as a Data Science and Machine Learning Vendor to Watch, 2022.
Our early success has been driven by customers using our real-time data analytic service for log analytics at scale. ChaosSearch is a perfect Elastic Stack (i.e. think ELK) replacement; our customers have eliminated the headache and cost of managing an Elasticsearch cluster without having to change user behavior.
But while we’ve proven successful at replacing Elasticsearch in the enterprise tech stack, that first use case is just one component of a larger vision: to transform "your" inexpensive, secure, and durable cloud object storage into an activated cloud data platform with infinite scale, disruptively low costs, and true multi-model data access.
Our vision is to give data-driven companies the ability to do any kind of analytics - full-text search, relational, or machine learning - on a single data lake with no data movement, no reindexing, no transformations, no ETL process, and no data retention trade-offs.
In 2021, we added support for Google Cloud Storage and introduced SQL capabilities with support for the Presto API Dialect. As the next step towards realizing our vision, we’re adding more capabilities to our platform in 2022 that will give our customers relational access to log data and unlock powerful new use cases for both log users and business intelligence (BI) teams.
Our blog post this week explores three use cases for relational access to log data and their potential to unlock new opportunities and insights for your business.
3 Use Cases for Relational Access to Log Data
Use Case 1: Bringing SQL/Relational Querying to Log Analytics Users
When we think about log analytics users, we’re thinking of ITOps, DevOps, and SecOps teams that depend on log analytics to troubleshoot cloud services, detect application performance issues, or hunt down cyber threats.
Until now, log analytics has been done primarily through full-text search - not relational capabilities. For log analytics users, gaining relational access to log data means unlocking new insights by combining SQL analytics and search querying. Log users will be able to:
- Leverage low-cost SQL access to log data at scale,
- Combine text search and relational querying to reveal hidden insights,
- Perform SQL joins of log data from multiple sources to reveal correlations that might have gone unnoticed in the past, and
- Create visualizations of log data using preferred business intelligence tools like Looker and Tableau.
With relational access to log data, SecOps teams can use SQL joins to create customized real-time alerts that trigger on logs generated by multiple sources (CDNs, IT infrastructure, network infrastructure, security infrastructure, etc.). DevOps teams could search for correlations between user experience data and application performance metrics to measure the impact of performance issues and prioritize bug fixes.
Use Case 2: Extending Log Analytics to Business Intelligence (BI) Teams
BI users include data analysts, LOB executives, and product teams who analyze data from a variety of sources to drive strategic decision-making.
The challenge for BI teams is that customer data exists in multiple silos (e.g. Salesforce CRM, web logs, marketing automation tools, Google Adwords, etc.), and it’s both difficult and expensive to search for correlations.
Companies solve this today by building and maintaining complex data pipelines, creating multiple transforms, and exporting data to multiple analytics tools - but it’s virtually impossible and certainly cost-prohibitive to maintain an accurate, 360-degree view of the customer while depending on these time and cost-intensive methods.
Adding SQL APIs and querying capabilities to our platform means that BI teams will be able to:
- Access even more data sources that drive insights into the customer experience,
- Push BI data from hundreds of different sources into a single data lake that supports search and relational querying,
- Perform SQL joins to combine customer data from multiple sources and create a single view of the customer, and
- Analyze customer data alongside log data to reveal hidden insights, and in real-time
With relational access to log data, companies with a focus on product-led growth can join A/B testing data with user metadata to delve deeper into customer experience and product usage, or combine product and user experience data to drive insights that grow product usage and revenue.
READ: Unlocking Data Literacy Part 1: How to Set Up a Data Analytics Practice That Works for Your People
Use Case 3: Enabling Multi-Dimensional Analytics on "Your" Data Lake
With log analytics and business intelligence happening in a single data lake that supports both relational and full text search, the last step to achieving true multi-model analytics is to enable machine learning queries.
Our final use case illustrates how both the users of BI tools and the users of log analytics software can combine text search, relational, and machine learning analysis to develop new insights.
In this unified data lake, analysts can use text search and machine learning detection to identify patterns and anomalies in their data. When a curious pattern or anomaly is discovered, data analysts can use the BI tools of their choice (e.g. Tableau, Looker, etc.) and visualization tools such as Kibana to engage with the data, analyze what they’ve found, and ask the right questions to uncover valuable new insights.
Finally, large companies with massive volumes of data consolidate data from hundreds of sources into a single data lake, analyze massive datasets with text search and machine learning algorithms to find patterns, then drill down to reveal insights using their preferred BI applications.
READ: Data Lake Architecture & The Future of Log Analytics
Looking ahead to 2022
The ChaosSearch team is excited about 2022. We’ve shown the value of a unified/streaming data lake for log analytics at scale but we’re especially psyched for activating more use cases for your data lake. ChaosSearch is enabling relational access to log data and empowering both log analytics and business intelligence teams to Know Better®. We’ve activated the data lake for log analytics at scale, and next we’re activating it for SQL and ML use cases. It’s been a thrilling, rocket ship of a ride. And we’re just getting started.
Ready to learn more?
Start Your Free Trial today, or View our On-Demand Demo to discover how the ChaosSearch platform delivers analytics at scale with data lake storage economics, no data movement, and unlimited data retention and queries.
Additional Resources
Read the Blog: Unlocking Data Literacy Part 2: Building a Training Program
Watch the Webinar: Modern Analytics: Data Lakes, Data Warehouses and Clouds
Check Out the Brief: Elasticsearch Replacement for Log Analytics at Scale
