Many thanks, Federico! I am taking this approach.
On Sat, Apr 20, 2024 at 10:41 AM Federico Leva (Nemo) nemowiki@gmail.com wrote:
Adding new checksum files may or may not be a big deal. If the snapshot hosts have enough memory to keep the files in cache a bit longer, so they don't need to be read back from disk, running new checksums may be very fast.
https://wikitech.wikimedia.org/wiki/Dumps has more information on the setup.
Il 20/04/24 03:40, Arthur D. Edelstein ha scritto:
In order to get the SHA-256 for the timestamp, I need to download each file and compute the hash.
I understand it's suboptimal, but if you're in a rush you can also use Toolforge and create a tool, a bit like https://dump-torrents.toolforge.org/ , to run sha256sum on the appropriate files (which are mounted even on the bastion host). I/O tends to be rather slow but may still be faster than your networking.
Best, Federico