Shared AiiDA environment

General Usage
1

Continuing the discussion from Multiple users on same RabbitMQ and Postgres database:

@sphuber I’m trying to set up something similar to what was proposed in the referenced post. I’m doing this by exposing shared services via docker compose including two aiidalab/full-stack instances for two users (8888 and 8889). My thinking is that by connecting everything across a shared network and using the same postgres and rabbitmq values during each profile setup should allow me to share data across the two containers. However, I am failing to do so, even after synchronizing the PROFILE_UUID.

Any suggestions? Happy to share my setup.

2

What is exactly the failure you get?

Note that in addition to PSQL and RabbitMQ, you’ll need to also share the disk-objectstore with some shared RW disk (to be then tested if there is no concurrency issue), or e.g. deploy a S3-like service (e.g. minio) and use the aiida-s3 plugin that Sebastiaan developed. Otherwise files are not going to be shared.

I also think that (not saying this should work already, but as a design) probably it’s better that

  1. every user has their own daemon and their own RabbitMQ
  2. if we go this way, we need to make sure that user 1 cannot inject new calculations in the DB and change maliciously the owner to user 2, and then the daemon of user 2 sees that and picks it up and runs it, with its key. This would effectively mean that AiiDA allows user 1 to impersonate user 2, even if the SSH keys are never shared, because you can ask the daemon of user 2 to run something.

I would at least get to the point where things work without considering 2, and that each daemon only picks up the jobs of the correct user (might need some changes in the code?)

Then, we can think at how to fix 2. I’m thinking at some kind of asymmetric key that is generated for each user, and any job submission request needs to be signed with this key, and the daemon of a user does not pick it up if the signature is invalid (so user 1, not having the private key of user 2, can inject in the DB a node claiming that it’s of user 2, but the signature will be invalid and user 2 will not run it).

@edan-bainglass in parallel to this solution, we should also investigate the sharing that I’ve been discussing with @geiger_j as well and probably a few others