You'll love Flink. I'd encourage using Apache Beam on top of Flink and use the unified API. That way you can take advantage of Java AND Python and Go (something that will be important for your teams)
https://flink.apache.org/ecosystem/2020/02/22/apache-beam-how-beam-runs-on-t...