Problem and why now
Currently proposed query syntax is "mostly DataFusion + some ClickHouse", which sounds weird and not unified. Since we are manually creating some query syntax by merging others'.
We propose considering using Ibis query syntax as input language.
Intended users and MVP
Users are any developer who want to use ASAPPlanner to come-up with plans.
MVP: I write a query in Ibis syntax and get a ASAP-styled plan from this module.
Inputs, outputs, and end-to-end outcome
Input: Ibis-styled query
Output: ASAP-plan
e2e effect does not change, this is just an interfacing stuff.
Simplest viable approach
No idea. I would assume there is some parser stuff in Ibis which we can leverage.
New concepts or interfaces requiring decisions
Need to decide if it is a good idea. I am not sure about this.
Alternatives and quality attributes
Do not understand this item
Acceptance behavior
Do not understand this item
Human decisions required
Need to decide if it is a good idea. I am not sure about this.
Pros: Logically this may sounds better, cuz now ASAP query syntax is not borrowed from its counterparts, but using a conceptually unified language.
Cons: (1) Not sure if it is necessary. (2) Ibis seems to be majorly using Python DataFrame APIs, though I remember SQL can also work. (3) Not sure Ibis itself is expressive enough to cover all what we need. Though it claims to be able to interface to different engines, but its coverage needs further investigation
Problem and why now
Currently proposed query syntax is "mostly DataFusion + some ClickHouse", which sounds weird and not unified. Since we are manually creating some query syntax by merging others'.
We propose considering using Ibis query syntax as input language.
Intended users and MVP
Users are any developer who want to use ASAPPlanner to come-up with plans.
MVP: I write a query in Ibis syntax and get a ASAP-styled plan from this module.
Inputs, outputs, and end-to-end outcome
Input: Ibis-styled query
Output: ASAP-plan
e2e effect does not change, this is just an interfacing stuff.
Simplest viable approach
No idea. I would assume there is some parser stuff in Ibis which we can leverage.
New concepts or interfaces requiring decisions
Need to decide if it is a good idea. I am not sure about this.
Alternatives and quality attributes
Do not understand this item
Acceptance behavior
Do not understand this item
Human decisions required
Need to decide if it is a good idea. I am not sure about this.
Pros: Logically this may sounds better, cuz now ASAP query syntax is not borrowed from its counterparts, but using a conceptually unified language.
Cons: (1) Not sure if it is necessary. (2) Ibis seems to be majorly using Python DataFrame APIs, though I remember SQL can also work. (3) Not sure Ibis itself is expressive enough to cover all what we need. Though it claims to be able to interface to different engines, but its coverage needs further investigation