Calculations get stuck in "created" state

General Usage
1

Dear all,
In the last few weeks I’m struggling a lot with calculations that get stuck in created state even for days. Sometimes after restarting the daemon and/or repairing processes they start again, many other times nothing changes and I should delete the calculation and launch it again. I was using aiida-core 2.6.1 and I tried updating it to 2.6.2, but still the same problem.
Do you have suggestions?
Thank you.

2

Hi @Davide_Bidoggia , could you please provide some more information. What is the output of verdi status?

3

Hi @sphuber, this is the output:

✔ version:     AiiDA v2.6.2
 ✔ config:      /home/bidoggia/.aiida
 ✔ profile:     bidoggia
 ✔ storage:     Storage for 'bidoggia' [open] @ postgresql://aiida_qs_bidoggia_1a762e6ea970b7891456fe9e1265d632:***@localhost:5432/bidoggia_bidoggia_1a762e6ea970b7891456fe9e1265d632 / DiskObjectStoreRepository: caa1c1ad4ce44ccfb08f0ca4f0b3cdda | /home/bidoggia/.aiida/repository/bidoggia/container
 ✔ broker:      RabbitMQ v3.9.13 @ amqp://guest:guest@127.0.0.1:5672?heartbeat=600
 ✔ daemon:      Daemon is running with PID 1137937
4

Since you are using RabbitMQ v3.9.13, did you follow the instructions to configure the consumer_timeout?

This is important for AiiDA to function properly, although it should not really explain the behavior you are describing. What would be very useful is some more information when you get a new process that you launch and gets struck in created. In that case, please check the daemon logs. It should be in ~/.aiida/daemon/log/aiida-{profile-name}.log. Search it for the pk of the stuck process. Also check the output of verdi process report and verdi node attributes. Also stop the daemon and run verdi process repair --dry-run just to confirm the task is actually missing.

Also, maybe there is just a problem with your daemon. Please check the ~/.aiida/daemon/log/circus-{profile-name}.log. Maybe your daemon workers are getting killed often and are restarted?