In any data collection or analysis, accuracy is paramount. But what about when you needed the metrics yesterday?!
The above is just an overdramatic way to say that data fidelity is fleeting. We live and operate within a world in constant flux, where Heisenberg’s uncertainty principle reigns supreme — for any data point (or particle), we can know the position, but not the momentum; or we can track the momentum and sacrifice our understanding of position.
Well, what if a query engine that captures multi-stream table data could update the data stream in real time? What if we could know both a data point’s position AND momentum, simultaneously? The latter is possible, if you’re working from a dynamic data set that updates as late-breaking developments emerge. Automatically. There are infinite applications to this concept, but let’s take the example of network monitoring, where alerts occur at the nanosecond.
Our 3-part Prometheus series explains in great detail how the Deephaven Core query engine can engage with Prometheus’s multi-dimensional data collection.
- In Part 1, we teach how to configure Deephaven to ingest Prometheus data via its REST API. Ingesting REST data in real time enables the user to ask and immediately answer questions like, “What’s the average value of this Prometheus query over the last 15 seconds?”
- In Part 2, we explain how Deephaven can ingest Prometheus data from alert webhooks. Pulling Prometheus’ alert webhooks directly into Deephaven enables the user to see the truest picture of this activity, even between data pulls.
For instance, if an IT department received a surge in 401 responses, one could logically assume that something happened with their customers’ login credentials. With the tandem of Prometheus + Deephaven Core, the user can diagnose the problem in real time, and address it (associating these web alerts with complementary data within Deephaven that confirm the cause to be login issues). - In Part 3, we explain how to combine separate data streams — one that tracks time-stamped metrics and a second that logs real-time web alerts. Deephaven especially excels at joining multiple time series data streams. By leveraging functionality like the as-of join, we can produce a single source of truth table that contains data from the Prometheus REST API and its alerts webhooks.
A single source of truth… in a table… that updates automatically and in real time? In terms of data analysis, visualization and calculation, well, we could do almost anything.
Pull the code from GitHub so that you can develop with Deephaven Core.
Source link
lol
