Log Streaming: What It Is and Why It Matters

You've probably heard the expression, "You don't know what you don't know." In other words, if you don't have access to all the usable data you need to inform business decisions, you won't even realize how much is missing from the equation. You need timely insights to empower your teams to work more effectively and deliver more value to users.

People around the world are spending an unprecedented amount of time working remotely and online. We use our personal devices to check in with work; we use our work devices in our homes. Considering all this, real-time monitoring and alerting is more critical than ever. In today's cloud-centric world, it's essential that engineering, architecture, and security teams have the tools they need to monitor their environments consistently, so they can respond to incidents and adapt to trends as close to real-time as possible.

What Is Log Streaming?

A log streaming service like Auth0's automatically saves logs to a third-party tool for storage and analysis, so you have the data you need to respond in near real-time to security threats, performance outages, and other issues. Event logs contain valuable insights about user activity, which makes them an important resource for improving operational health and security. Here are some key functionalities enabled by event logs:

• Troubleshooting connectivity problems
• Identifying configuration areas that could use performance improvements
• Identifying where and why service disruptions occurred
• Exposing metrics to identify a target audience
• Pinpointing prime usage times by region
• Identifying bad actors by IP address
• Detecting and optimizing user behavior patterns
• Enabling debugging and integration setup by exposing where customers are experiencing pain and where the flow may be broken

Auth0's log streaming service lets you stream events to a host of integration partners, such as Amazon EventBridge, or any given URL. Log streaming allows you to:

• Export logs to a tool or service you already use
• React to events, such as changed passwords or new registrations, with your own business logic by sending events to custom webhooks (for instance, you can use log streaming to drive automated marketing campaigns upon signup)
• Send events to Amazon EventBridge for processing with AWS Lambda or additional data pipelines

Without log streaming, you don't have the information you need to respond promptly when intervention is called for.

See Critical Identity Information in Near Real-Time

Auth0's log streaming service gives you access to mission-critical information — identity activity, identity security, account operations, and more — in near real-time. With log streaming, you can continuously export tenant log events to your existing monitoring and alerting dashboards, streamlining operations, and realizing deeper insights into user behavior.

Auth0 log streaming works by sending micro-batches of streaming events as they are generated in Auth0, giving customers up-to-date information about their Auth0 tenant. Implementing log streaming delivers scalable, resilient, and low-latency access to your Auth0 events.

Make informed development decisions

Tracking trends over time allows you to make informed decisions about product development and system architecture. For example, authentication data can be used to determine how to prioritize development resources: If more users are authenticating from mobile devices, mobile development goes to the top of your priority list.

Another example is tracking peak login times by geographic location to help system architecture teams determine when and where to scale up resources. The ability to scale rapidly allows you to change your IT deployments to support your current needs, so your developers can focus on adding value instead of tinkering endlessly with your infrastructure. Fast scalability is also crucial for delivering a smooth user experience in response to short-term events like Election Day or the World Cup.

Respond to performance and security incidents immediately

Tracking identity information helps you detect and respond to security incidents and performance issues before they do too much damage. Equipped with the right information, your security team can configure thresholds and set up alerts to notify them when suspicious events or patterns are detected, enabling faster response times.

Whether it's general operational health or security, teams often rely on a rapid visual analysis to differentiate between normal and abnormal states and determine when they need to deep-dive to investigate and resolve potential issues. For instance, a massive spike in the number of unsuccessful login attempts could indicate an ongoing credential stuffing attack — one of the most common and damaging ways you can expect attackers to target your infrastructure.

Event logs also yield insights into performance issues and outages, so you can understand what went wrong, who's affected, and how to fix things before your customers experience pain.

Manage Log Streaming With Auth0 Marketplace

An inherently extensible platform like Auth0 enables users to customize identity flows and integrate with third-party solutions to meet specific business needs — now and in the future. Extensibility describes a system's capacity to support future growth via new functionalities and use cases, as well as the level of effort required to implement those extensions.

Opening up our platform to third-party, partner-built integrations via Auth0 Marketplace accelerates innovation for Auth0, our customers, and our partners. The ecosystem gives our customers the tools they need to solve the range of use cases they encounter. With Marketplace, developers get more integrations to support more use cases, and vendors have more opportunities to capture business and deliver value. For vendors who haven't yet built integrations with Auth0, Marketplace creates that opportunity.

To provide our customers with enhanced security and operational and monitoring insights, Auth0 has partnered with the following companies:

Amazon EventBridge

Amazon EventBridge makes it easy to connect applications using data from your own apps, integrated SaaS apps, and AWS services. EventBridge delivers a stream of real-time data from event sources like Datadog, Pagerduty, or Zendesk, then routes that data to targets like AWS Lambda.

With EventBridge, you can set up routing rules to determine where to send your data to build application architectures that react in real-time to all of your data sources. EventBridge takes care of event ingestion and delivery, security, authorization, and error handling for you, making it easy to build event-driven applications.

Azure Event Grid

Azure Event Grid simplifies event-based apps by providing a single service for managing routing of all events from any source to any destination. Designed for high availability, consistent performance, and dynamic scale, Event Grid lets you focus on your app logic rather than your infrastructure.

Event Grid simplifies event delivery by allowing you to build scalable serverless applications, microservices, and distributed systems. By using Event Grid to capitalize on guaranteed event delivery and high availability on the cloud, you can build better-performing, more reliable applications through reactive programming. Finally, Event Grid allows you to focus more resources on innovation: Connecting multiple possible sources and destinations of events helps you develop richer application scenarios.

Datadog

Datadog (NASDAQ: DDOG) is a monitoring and analytics platform for cloud applications that has become a critical enabler for many companies. Engineering, infrastructure, and security teams use Datadog to unify data from servers, containers, databases, and third-party services to make the stack entirely observable.

With this integration, Auth0 and Datadog combine resources to deliver mission-critical identity information to our joint customers. The integration allows customers to incorporate Auth0 logs into their existing monitoring and alerting dashboards without additional engineering work. Users can collect and visualize Auth0 data in order to identify trends — all without extra development time.

With Auth0-Datadog integration, engineering teams can visualize error rates and traffic data. Security teams can visualize authorization traffic and set up alerts for high-risk actions. Infrastructure teams can ensure stability by visualizing performance metrics and traffic data. For all of these use cases, identity data furnishes crucial insights, and the Auth0-Datadog integration helps you make full use of that information.

Perch Security

Perch Security allows you to add managed threat detection and SIEM (security information and event management) without changing your existing stack. Perch Security is fast and simple to deploy, supports multitenancy and third-party integrations, and is backed by a 24/7 U.S.-based security operations center.

With Perch, you receive alerts about atypical and potentially malicious events that happen within your Auth0 environment. You can also generate reports based on the log data collected from your Auth0 environment and retain those logs for future reference and to help satisfy auditory or regulatory requirements.

Splunk

Splunk is a data platform that allows companies to analyze data in any structure, from any source, across any timescale. Not only does Splunk make it easy for companies to understand the health of their operating system in terms of performance and traffic, but it also offers robust SIEM and SOAR (security orchestration, automation, and response) capabilities via Splunk Enterprise Security and Splunk Phantom. These capabilities encompass monitoring, detection, security, threat detection, and workflow automation.

Auth0's event data provides rich contextual information to help our customers make informed decisions about system architecture and development. The Splunk integration enables Auth0 to deliver mission-critical identity information in a fast, scalable way. Our shared customers can easily leverage this information to make more informed decisions. The integration also allows for easy visualization of security and operational signals from Auth0 within Splunk Cloud and Splunk Enterprise since customers can now incorporate Auth0 event logs into their existing Splunk environments with minimal engineering work.

In addition, the integration allows customers to build workflows to improve security posture by leveraging Auth0's event logs and Splunk's SOAR capabilities. For example, security teams can create a playbook within Splunk Phantom for Auth0's breached password detection event logs to automatically block an account and force the user to reset their password, all without any manual interaction. Security teams can also leverage security-specific events to automatically trigger proactive investigation and mitigation.

As part of this integration, Auth0 has created a custom Splunk dashboard that automatically visualizes critical security signals coming from Auth0. This allows security teams to monitor authorization traffic, analyze anomalies, and set up alerts for high-risk actions with greater confidence.

Meeting Tomorrow’s Identity Needs

With Auth0 Marketplace, we're recruiting best-in-class technology partners to innovate on our platform because we know that's the best way to give you the tools you need to solve whatever identity puzzles lie ahead.

To learn how to integrate with our log streaming partners, start here.