Support for Timestral and Tirex-2 in AINode

[kotselidis]

Hi All, I have added support for Timestral and Tirex-2 in AINode for a side project I am currently doing. Are you interested in integrating them into main? If yes, I can open the PRs.

[SpriCoder]

Hi, thanks for your contribution! We'd be happy to have these integrations in AINode, so please feel free to open the PRs!

It's great to hear that you're using our software in your project. We'd love to learn more about your use cases and scenarios if you're willing to share!

[kotselidis]

Great, thanks. I will prepare and open the PRs soon.
I am leading the TornadoVM project (www.tornadovm.org) that accelerates Java on GPUs. On top of TornadoVM I have built jitCast (https://github.com/beestream-ai/jitCast)which is framework that runs Java-based GPU-accelerated time-series models such as Chronos, Timestral and Tirex. I have integrated TornadoVM and Jitcast inside ApacheIoT's data node and I am running experiments to see if its faster to do the AI inference in-node versus shipping data to the AI node that runs Pytorch. I want to understand whether in-site in-node GPU accelerated data processing and AI inference beats the round-trip to the AI node. Would like to hear your expert thoughts on this idea

[SpriCoder]

First of all, welcome to our community! That sounds really interesting! One of the key motivations behind AINode is to reduce data movement overhead by bringing AI inference closer to the data, so your experiments align well with this direction. I'm looking forward to hearing about your results, and we'd be happy to explore further collaboration in different areas! Also, feel free to reach out to me anytime once you've opened the PRs.

[kotselidis]

Please drop me an email to continue offline

[SpriCoder]

Attach me here: spricoder@apache.org

[RkGrit]

Hi professor, thanks for adding support for Timestral and Tirex-2! We’d be very happy to welcome your contributions to AINode.

I’m working on training and inference for time-series foundation models, and I’d love to learn more about your project and exchange ideas and experiences. It would be great to explore opportunities to collaborate, both on AINode and on related research. Looking forward to further discussions!

[CRZbulabula]

Hi @kotselidis, thanks for sharing the additional context. The inference-placement experiment sounds useful. It would help to review the AINode model adapters separately from the DataNode/TornadoVM integration.

One point worth checking before merging is the distinction between inference-code licenses and pretrained-weight licenses:

• TiRex-2: Both the public inference code and model weights are Apache-2.0, so this looks like a good candidate for an AINode PR, subject to the usual dependency and attribution review.
• Timestral: The inference code is Apache-2.0, but the weights use a Modified MIT license. It withholds authorization when the company’s or employer’s global consolidated revenue for the preceding month exceeds USD 20 million, including for derivatives and combined works. This is a material restriction beyond standard MIT. We should therefore avoid committing to bundled weights or making them a required dependency. An optional integration using user-supplied weights could be assessed separately against the ASF optional-dependency rules, with PMC/ASF Legal guidance. The Apache-2.0 licenses of jitCast and TornadoVM do not change the weights’ terms.
  For the performance experiment, embedded inference could reduce RPC and serialization overhead. However, comparing embedded TornadoVM against remote PyTorch changes both the execution engine and deployment location. A co-located AINode baseline would help separate those effects.

Using the same checkpoints, precision, context lengths, forecast horizons and batch sizes, it would be useful to measure cold-start and warmed-up latency, throughput, forecast accuracy, and the impact on concurrent database reads and writes. Separating queueing, serialization/RPC, host-to-device transfers and model execution would make the results especially informative. AINode’s process isolation and independent resource scaling also matter when assessing the trade-offs.

There is also a runtime compatibility question: jitCast currently requires JDK 22+, while IoTDB’s Java baseline is 17. That deserves a separate design discussion alongside GPU resource management and failure handling.

Please feel free to open the model PRs separately, including the exact upstream package/checkpoint versions and their licenses. That would let us review TiRex-2 while resolving the Timestral licensing question independently.

[kotselidis]

I also thought about the Timestral license problem and the solution I implemented was to not to download the weights on AINode launch time but instead force the user to explicitly download them. This added some extra code that guards against potential inferences while the weights are still downloading. You can check the PR to see if the solution fits your architecture.

Regarding jitCast and TornadoVM (both Apache 2.0), I think those will not need any integration (code-wise) since they are just jars that can be loaded. I do not want to change at all the code of the DataNode in order to use jitCast. The PRs for both Timestral and TiRex are open so please feel free to assess them and request any changes you seem fit.

[CRZbulabula]

Thanks for opening #19272 and #19273 and clarifying the deployment approach. Deferring the Timestral download until an explicit LOAD MODEL addresses the unsolicited startup download, and the inactive/activating states make the proposed lifecycle easier to assess. I am reviewing the implementation details in the PRs and will keep code-specific feedback there.

The weights' licensing conditions still apply, though. Please link the exact checkpoint license in the user-facing setup instructions and explain the revenue restriction before users initiate the download; executing LOAD MODEL does not itself establish eligibility. Keeping the weights outside our distributions and the feature optional gives us a concrete proposal to assess under the ASF optional-dependency policy, but we should resolve whether this download workflow is acceptable with the PMC/ASF Legal before treating that question as settled. The existing Moirai2 behavior should be reviewed separately as well.

Thanks also for clarifying that jitCast can use existing extension points without DataNode source changes. Could you share a minimal adapter/invocation example and the exact JDK, TornadoVM backend dependencies, and JVM flags used? jitCast currently targets Java 22, so the deployment requirements still need to be explicit alongside IoTDB's Java 17 baseline. GPU memory ownership, cancellation, and resource cleanup are worth testing when inference shares the database process.

For the performance study, a comparison of embedded jitCast, co-located AINode, and remote AINode with matching checkpoints, precision, and workloads would be particularly useful. Warmed-up inference latency together with the impact on concurrent reads and writes would help separate execution-engine gains from data-transfer savings.

[kotselidis]

Thanks for the reviews and your comments. I will try to address all of them in due time.
Regarding jitCast, I will prepare an example and send it over. I hope this will not be a PR but rather a guide on how to use it as an external module. Regarding min JDK, I think I can not do anything about it since jitCast is based on TornadoVM which uses Project Panama JDK > 21.

Thanks also for the suggestions regarding the performance analysis. We do not have an IoT cluster in our premises; we can only test two remote servers with different Data/AI node colocations. I think the challenge is to separate where the performance differences come from (difference in perf between PyTorch and JitCast/TornadoVM, difference in local/remote execution, and benchmarks used).

Regarding benchmarks, I use TimeStral, Chronos and Tirex for inference and benchmarks from https://github.com/thedatumorg/TSB-UAD

Do you have any other suggestions?