Technically, you can make the tool open-source and keep the source code secret. That solves the maintenance problem (others who get access can legally modify). Of course, you'd have to trust everyone with access to the files to not publish them which they would be technically entitled to (unless there is some NDA-like mechanism).
Transparency and auditability wouldn't be fulfilled just by making the code public, anyway; they need to be solved by tool design (keeping logs, providing feedback options for the users, trying to expose the components of the decision as much as possible).
I'd agree with Bawolff though that there is probably no point in going to great lengths to keep details secret as creating a similar tool is probably not that hard. You can build some assumptions into the tool which are nontrivial to fulfill outside Toolforge (e.g. use the replicas instead of dumps) to make running it require an effort, at least.