Hello !
I am running into some problems running one of my pipelines with Aiida. Everything was working fine with this code (I ran successfully until the end several times) until at some point all my processes were ‘Waiting’ for one day or two and when I tried to kill them, I had ‘Error: Process<…> is unreachable’.
I found someone that said to solve the issue they deleted the nodes (Process unreachable after stopping script · Issue #3268 · aiidateam/aiida-core · GitHub), and that is what I did.
But now, whenever I do a ‘verdi run’ with the same code as before that was working and now it doesn’t go past the ‘Created’ process state…
Do you have any suggestion or idea?
Thank you !
Vincent
Hi Vincent, this shows there probably is a problem with your daemon and it can typically be fixed by simply restarting it with verdi daemon restart --reset. It might take a bit for the processes to start running again depending on how many processes you have in total. If you still have problems, stop the daemon and run verdi devel rabbitmq tasks analyze --fix and start the daemon again.
Hello,
The daemon is actually not answering anymore… ‘verdi status’ gives me ‘daemon: Call to the circus controller timed out’. ‘verdi daemon stop’ gives me ‘Waiting for the daemon to shut down… TIMEOUT’…
Is there a way to force the daemon to stop?
Thank you for your help !
Vincent
Hi Vincent, sorry for the late response. Normally when you run verdi daemon stop it will eventually stop. Even though the command itself may say it timed out. If it really doesn’t stop. You should run ps aux | grep start-circus. This should show something like:
user 8041 0.9 1.1 515520 89400 ? Sl 11:01 0:00 /path/bin/verdi -p main daemon start-circus 1
user 8135 0.0 0.0 6392 720 pts/3 S+ 11:01 0:00 grep --color=auto start-circus
The second column of the first row contains the process ID of the daemon process (8041 inthis example). You can kill it using kill -9 8041. Then you can restart the daemon with verdi daemon start.
1 Like