On Tue, 1 Jun 2021 at 10:53, Neil Girdhar wrote:

On Tue, Jun 1, 2021 at 5:39 AM Oscar Benjamin wrote:

...

On Tue, 1 Jun 2021 at 05:16, Neil Girdhar wrote:

...

Hi Oscar,

The problem that the original poster was trying to address with additional syntax is the automatic naming of symbols. He wants to omit this line:

x = symbols("x")

You're right that if you have many one-character symbol names, you can use a shortcut, but this benefit is lost if you want descriptive names like:

momentum = symbols('momentum')

He is proposing new syntax to eliminate the repeated name. The function approach specifies each name exactly once. This is one of the benefits of JAX over TensorFLow.

Second, the function approach allows the function to be a single object that can be used in calcuations. You might ask for:

grad(equation, 2)(2, 3, 4 5) # derivative with respect to parameter 2 of equation evaluated at (2, 3, 4, 5)

With the symbolic approach, you need to keep the equation object as well as the symbols that compose it to interact with it.

This makes more sense in a limited context for symbolic manipulation where symbols only represent function parameters so that all symbols are bound. How would you handle the situation where the same symbols are free in two different expressions that you want to manipulate in tandem though?

In this example we have two different equations containing the same symbols and we want to solve them as a system of equations:

p, m, h = symbols('p, m, h') E = p**2 / 2*m lamda = h / p

E1 = 5 lamda1 = 2 [(p1, m1)] = solve([Eq(E, E1), Eq(lamda, lamda1)], [p, m])

I don't see a good way of doing this without keeping track of the symbols as separate objects. I don't think this kind of thing comes up in Jax because it is only designed for the more limited symbolic task of evaluating and differentiating Python functions.

This is a really cool design question.

One of the things I like about JAX is that they stayed extremely close to NumPy's interface. In NumPy, comparison operators applied to matrices return Boolean matrices.

I would ideally express what you wrote as

def E(p, m): ...

def lamda(h, p): ...

def f(p, m): return jnp.all(E(p, m) == E1) and jnp.all(lamda(h, p) == lamda1)

p1, m1 = solve(f)

So how does solve know to solve for p and m rather than h? Note that I deliberately included a third symbol and made the parameter lists of E and lamda inconsistent. Should Jax recognise that the 2nd parameter of lamda has the same name as the 1st parameter of E? Or should symbols at the same parameter index be considered the same regardless of their name? In Jax everything is a function so I would expect it to ignore the symbol names so that if args = solve([f1, f2]) then f1(*args) == f2(*args) == 0. This is usually how the API works for numerical rather than symbolic root-finding algorithms. But then how do I solve a system of equations that has a symbolic parameter like `h`? -- Oscar

Oh, and I see you're a sympy developer! I hope I'm not coming across as critical in any way. I love that sympy exists and thought it was a really cool project when I first learned about what it can do. Perhaps we should move our discussion to a "Github discussion" under the sympy Github? We might be able to build a toy example that takes functions, inspects the signature, and converts it into a traditional sympy expression. Then we could examine a variety of examples in the docs to see whether the function-based symbols are easier or harder to use in general. Best, Neil On Tue, Jun 1, 2021 at 6:47 AM Neil Girdhar wrote:

On Tue, Jun 1, 2021 at 6:28 AM Oscar Benjamin wrote:

...

On Tue, 1 Jun 2021 at 10:53, Neil Girdhar wrote:

...

On Tue, Jun 1, 2021 at 5:39 AM Oscar Benjamin wrote:

...

On Tue, 1 Jun 2021 at 05:16, Neil Girdhar wrote:

...

Hi Oscar,

The problem that the original poster was trying to address with additional syntax is the automatic naming of symbols. He wants to omit this line:

x = symbols("x")

You're right that if you have many one-character symbol names, you can use a shortcut, but this benefit is lost if you want descriptive names like:

momentum = symbols('momentum')

He is proposing new syntax to eliminate the repeated name. The function approach specifies each name exactly once. This is one of the benefits of JAX over TensorFLow.

Second, the function approach allows the function to be a single object that can be used in calcuations. You might ask for:

grad(equation, 2)(2, 3, 4 5) # derivative with respect to parameter 2 of equation evaluated at (2, 3, 4, 5)

With the symbolic approach, you need to keep the equation object as well as the symbols that compose it to interact with it.

This makes more sense in a limited context for symbolic manipulation where symbols only represent function parameters so that all symbols are bound. How would you handle the situation where the same symbols are free in two different expressions that you want to manipulate in tandem though?

In this example we have two different equations containing the same symbols and we want to solve them as a system of equations:

p, m, h = symbols('p, m, h') E = p**2 / 2*m lamda = h / p

E1 = 5 lamda1 = 2 [(p1, m1)] = solve([Eq(E, E1), Eq(lamda, lamda1)], [p, m])

I don't see a good way of doing this without keeping track of the symbols as separate objects. I don't think this kind of thing comes up in Jax because it is only designed for the more limited symbolic task of evaluating and differentiating Python functions.

This is a really cool design question.

One of the things I like about JAX is that they stayed extremely close to NumPy's interface. In NumPy, comparison operators applied to matrices return Boolean matrices.

I would ideally express what you wrote as

def E(p, m): ...

def lamda(h, p): ...

def f(p, m): return jnp.all(E(p, m) == E1) and jnp.all(lamda(h, p) == lamda1)

p1, m1 = solve(f)

So how does solve know to solve for p and m rather than h?

Because those are the parameters of f.

...

Note that I deliberately included a third symbol and made the parameter lists of E and lamda inconsistent.

Should Jax recognise that the 2nd parameter of lamda has the same name as the 1st parameter of E? Or should symbols at the same parameter index be considered the same regardless of their name?

It doesn't need to because the same variable p (which will ultimately point to a tracer object) is passed to the functions E and lamda. It's not using the names.

...

In Jax everything is a function so I would expect it to ignore the symbol names so that if args = solve([f1, f2]) then f1(*args) == f2(*args) == 0.

This is usually how the API works for numerical rather than symbolic root-finding algorithms. But then how do I solve a system of equations that has a symbolic parameter like `h`?

I assumed that h was a constant and was being closed over by f. If h is a symbol, I think to stay consistent with functions creating symbols, we could do:

def f(p, m, h): return E(p, m) == E1 and lamda(h, p) == lamda1

def g(h): return solve(partial(f, h=h))

g is now a symbolic equation that returns p, m in terms of h.

By the way, it occured to me that it might be reasonable to build a system like this fairly quickly using sympy as a backend.

Although, there are some other benefits of Jax's symbolic expression-builder that might be a lot harder to capture: Last time I used sympy though (years ago), I had a really hard time making matrices of symbols, and there were some incongruities between numpy and sympy functions.

Best,

Neil

...

-- Oscar