The bottleneck is the MySQL database. Up to now I have
used the standard
docker-compose setup. With the Java program I can write only one item
per second.
I suspect that I have found the solution. The trick is to put the insert
operations into a transaction.
In a simple experiment with 1,000 item it took 760 ms per item. Within a
transaction an item could be inserted less than 3 ms. What a speed-up!
Now the insertion of my 40 million items seems to be doable. :-)
After I have cleaned up my demo code a bit I am going to share it via
GitHub.
Jesper