Python variables now automagically work in Deephaven queries! | Deephaven

Deephaven’s query language provides a very powerful way to manipulate and analyze static and real-time data. Part of Deephaven’s power is the ability to use Python functions and variables directly in your queries. As of Deephaven v0.16, using Python in Deephaven queries is even easier.

Let’s look at a few examples.

To begin, let’s create some data to work with:

from deephaven import empty_table

source = empty_table(10).update("X = ii")

Let’s define a variable a in the global Python scope. Once variable a exists, it can be used in a query string. You can see a being used in both a where and an update. Very cool.

a = 3
t = source.where("X > a").update("A = a + 3")

Maybe you want to create a function to encapsulate some reusable query logic. Deephaven has you covered.

def query_logic(table, b):
    return table.where("X > b").update("B = b + 9")

t1 = query_logic(source, 1)
t2 = query_logic(source, 2)

What about something more complex like a set comprehension? That works, too.

vals = [1, 6, 2]
results = {v : source.where("X > v").update("V = sqrt(v)") for v in vals}
t1 = results[1]
t2 = results[2]
t6 = results[6]

Remember that in Python functions are first class objects and behave like any other Python objects, such as ints or lists. This allows functions to be used in queries just like any other variable.

import math

def f1(x):
    return math.sqrt(x) + 3

f2 = lambda x: x + 3

t = source.where("(double) f1(X) > 2").update("F = (int) f2(X)")

In Python, the LEGB rule (local / function scope, enclosing / nonlocal scope, global / module scope and built-in scope) determines how variable names are resolved. Deephaven supports the local / function scope and the global / module scope. The unsupported built-in scope is mostly Python keywords, such as print, that you should not need to access directly in a query string. To make the enclosing / nonlocal scope work, use the nonlocal keyword to bring the variable into the local scope.

def f_gen(t, v):
    def inner(x):
        nonlocal v
        return t.where("X > x").update("V = sqrt(v)")

    return inner

f = f_gen(source, 3)
t = f(4)

What will you do with Deephaven’s seamless Python integration? Tell us on Slack.