Anomalo

We appreciate that we can choose exactly what we want to monitor and how to do it, thanks to the tool's ability to build our own tests. Anomalo doesn't provide a clear understanding of the costs involved in running validations on the Data Warehouse. Anomalo has improved cooperation and involvement among our teams and provided us with valuable insights that even non-technical business users can understand and apply.

Integration with AWS Athena and Glue works well. There's a good API for running operations and getting stats. Visuals are very good for non-technical users. The staff is helpful. However, they're relatively early in development, and there are problems with nested structs and objects (not arrays). Possibly they're trying to do too many integrations at once, and some can be buggy. It helps us with cell-level data quality issues, which is beneficial for our customers.

With Anomalo, we can easily focus on improving coverage for our important tables and passing metric checks. We can dive into any statistics to clarify what our priorities should be. This also gives us visibility into tables that impact our SLA by checking their data arrival times in the dashboard. We've recently started using Anomalo to identify data bottlenecks and determine the root cause quickly. We'd like proper documentation for its features so we can become more familiar with the platform. We integrate Anomalo with our customer data warehouse to perform seamless data quality checks and regulate alerts for anomalies. Anomalo's Pulse dashboard is brilliant for unsupervised detection, where even slight changes in our customer datasets are caught. We can easily create dynamic testing approaches regardless of the data quality conditions. Investigating production issues is also accurate because we can expand on any data structure in the Pulse dashboard. It also effectively prevents abnormal schema changes and dataset omissions that usually cause unwanted disruptions.

The interface is clear and intuitive. The learning curve is low; using Anomalo doesn't require hours of training. The data graphs are beautiful and very helpful for analyzing data and spotting anomalies. Setting up a new table takes just a minute, and you get 8 built-in checks right away. There's also a wide range of custom checks. However, in its simplicity, Anomalo sometimes appears limited and lacks certain functionalities. For instance, there are no tools for supporting multiple teams, like sending notifications for the same table to different destinations, adding comments by different teams, or tagging tables. There's also no way to control the amount of data scanned to manage costs, or to decide if a table is worth keeping in Anomalo. Thanks to Anomalo and its transparent charts from Data Freshness and Data Volume sections, we've been able to repeatedly detect data gaps for various days, including historical ones. We can also clearly see anomalies related to the amount of processed data for a table on a specific day. Anomalo gives us, the Data Engineers team, a sense of control and security.

It's the best tool for data quality and validation to engage both business users and developers. The Root Cause Analysis features give us better arguments when talking to developers and reduce the time we spend digging into the causes of data quality problems. Customization needs more attention. For use cases beyond the original tool's scope, things can become unfeasible. Also, it doesn't give a clear view of the collateral costs of running validations in the Data Warehouse, which can be an issue in large projects with big tables. We use it to detect data quality issues and communicate them to engineering and business users. It helps by fostering more engagement and collaboration among product, data, and engineering teams to interpret and resolve data incidents.

Anomalo is an excellent tool for keeping an eye on overall data quality. Beyond the standard metrics like freshness and availability, it's great to be able to set up custom checks. Some standout features are the anomaly history, which is useful for understanding main variations in column statistics, and the ability to configure alerts using multiple columns, tables, or SQL. One limitation is that you can't set up a BigQuery wildcard table as a single table. For example, with Google Analytics and BigQuery, data is stored in a wildcard table, and monitoring the quality of user interactions on our website is crucial, so having this feature would be helpful. We typically use Anomalo for monitoring our key tables with critical financial data. Ensuring the quality of our sales data is vital because errors can harm the company. We also use it for data that feeds our ML models to assure quality and act quickly on any issues.

As an admin, setting up Anomalo on our existing infrastructure was really simple, and we got it running quickly. This meant we started seeing value almost immediately. We integrated it with our k8s setup and it was fairly straightforward. Anomalo consistently finds small issues in our datasets and helps us pinpoint the cause even before vendors notice. We've flagged issues upstream multiple times when the data producer wasn't aware yet. We've tried many tools for this problem in the past, and Anomalo has outperformed all our homegrown solutions in both adoption and value. The UI is extremely user-friendly and great to work with, making it easier for data analysts and non-technical folks to get on board and set up custom checks without writing code. Anomalo's team has been incredibly responsive and helpful. One downside is that checks aren't very configurable. If a dataset has an expected sudden spike in rows, Anomalo will alert for 2-3 days until it learns the new normal. Also, the out-of-the-box tests can be pricey in runtime and computing metrics; they scan all columns, even ones not used anywhere, resulting in wasted work. Alerts can be noisy, and it's up to the team to have a process to address each one. Anomalo helps us verify if our datasets are actually correct and match our expectations. This gives data teams more control over our datasets and allows us to scale the number of datasets we handle without needing more engineers.

Rather than only checking metadata like file arrivals or basic row counts, Anomalo uses unsupervised machine learning on the data records themselves. It's powerful for catching unexpected data errors without manual rules. However, its deep-record queries can inflate your cloud warehouse costs, it doesn't support real-time streaming, and it has trouble with alert fatigue on seasonal data patterns. Anomalo addresses undetected data corruption and does away with manual SQL check maintenance. This helps me by saving engineering time on pipeline debugging, preventing broken executive dashboards, and keeping toxic or corrupted data away from your generative AI and machine learning models.

Anomalo is great at automatically spotting and diagnosing data issues through machine learning, with no manual setup needed. Its smooth integration with data warehouses and easy-to-navigate interface make it a strong tool for maintaining proactive, dependable data quality. Some users might find that Anomalo's ML models need some adjustment to perfectly fit their business needs. Also, teams new to data quality platforms might face a learning curve with its advanced features. Anomalo tackles the challenge of ensuring data quality by automatically identifying anomalies and diagnosing problems across datasets. This proactive method helps keep trust in data, cuts down manual work, and ensures reliable insights for better decisions.

The main advantage for me is the time I save. Rather than manually tracking datasets or waiting for someone to spot a problem in a dashboard, Anomalo catches unusual patterns on its own. It has enabled us to identify data issues before they affect reporting, which has boosted trust in our data. I'd love to see more customization options in certain dashboards and reporting views. What's currently available works well, but added flexibility would allow teams to tailor the experience to their specific needs. Anomalo helps us with data reliability and quality challenges. By automatically detecting anomalies and issues, it speeds up troubleshooting and prevents bad data from reaching our dashboards and business users. This means we spend less time on firefighting and have more confidence in our analytics.