Microsoft has announced and is open-sourcing a new data processing system called Drasi that can detect and react to changes in complex systems.
This new project “simplifies the automation of intelligent reactions in dynamic systems, delivering real-time actionable insights without the overhead of traditional data processing methods,” Mark Russinovich, CTO, deputy chief information security officer, and technical fellow at Microsoft Azure, wrote in a blog post.
It watches for events in logs and change feeds without having to copy data to a central data lake or continuously querying data sources. Developers can use define which changes they want to track, and then Drasi decides if changes should trigger an action.
“If they do, it executes context-aware reactions based on your business needs. This streamlined process reduces complexity, ensures timely action while the data is most relevant, and prevents important changes from slipping through the cracks,” Russinovich explained.
Drasi can be boiled down into three components: Sources, Continuous Queries, and Reactions.
Sources connect to data sources like application logs, database updates, or system metrics and continuously monitor for critical changes.
Continuous Queries continuously evaluate incoming changes based on some predefined criteria.
Reactions are made when a change completes a continuous query, and can include tasks like sending alerts, updating other systems, or performing remediation steps.
According to Microsoft, developers building event-handling mechanisms have often turned to multiple tools, resulting in fragmented and complex architectures. Additionally, a lot of change detection tools don’t have real-time capabilities and instead rely on batch processing, data collation, or delayed event analysis.
“For businesses that need immediate reactions, even these slight delays can lead to missed opportunities or risks. In short, there is a pressing need for a comprehensive solution that detects and accurately interprets critical events, and automates appropriate, meaningful reactions,” Russinovich wrote.
The project has been submitted to the Cloud Native Computing Foundation (CNCF) as a Sandbox project, meaning that if it’s accepted, it will get support, guidance, and resources from the organization. It is being licensed under the Apache 2.0 license.