Usage of AiiDA-QE in multiple conda environments

General Usage
,
1

Hi,

I have different projects that involve the usage of AiiDA for which I have created separate Conda environments. I would like to report an issue that I am facing. I submitted a PwWorkchain calculation using the sscha environment. The calculation ran successfully, and I would like to retrieve the total energy of my system from this calculation. I can do this without any problem in the aiida environment (the one where I set up my AiiDA profile).

(aiida) gjoalland@ctimac001:~$ verdi shell
Python 3.9.21 | packaged by conda-forge | (main, Dec  5 2024, 13:51:40) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.18.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: n=load_node(18791)

In [2]: n.outputs.output_parameters.dict.energy
Out[2]: -28880.263864815

However, when I follow the same procedure in the sscha environment, it does not work:

(aiida) gjoalland@ctimac001:~$ conda activate sscha
(sscha) gjoalland@ctimac001:~$ verdi shell
Python 3.9.21 | packaged by conda-forge | (main, Dec  5 2024, 13:51:40) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.18.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: n=load_node(18791)

In [2]: n.outputs.output_parameters.dict.energy
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[2], line 1
----> 1 n.outputs.output_parameters.dict.energy

File ~/work/setup/aiida-core/src/aiida/orm/nodes/node.py:797, in Node.__getattr__(self, name)
    792     warn_deprecation(
    793         f'`{kls}.{name}` is deprecated, use `{kls}.base.links.{new_name}` instead.', version=3, stacklevel=3
    794     )
    795     return getattr(self.base.links, new_name)
--> 797 raise AttributeError(name)

AttributeError: dict

Even restarting the daemon does not seem to help:

(sscha) gjoalland@ctimac001:~$ verdi daemon restart
Profile: gjoalland
Stopping the daemon... OK
Starting the daemon with 1 workers... OK
(sscha) gjoalland@ctimac001:~$ verdi shell
Python 3.9.21 | packaged by conda-forge | (main, Dec  5 2024, 13:51:40) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.18.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: n=load_node(18791)

In [2]: n.outputs.output_parameters.dict.energy
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[2], line 1
----> 1 n.outputs.output_parameters.dict.energy

File ~/work/setup/aiida-core/src/aiida/orm/nodes/node.py:797, in Node.__getattr__(self, name)
    792     warn_deprecation(
    793         f'`{kls}.{name}` is deprecated, use `{kls}.base.links.{new_name}` instead.', version=3, stacklevel=3
    794     )
    795     return getattr(self.base.links, new_name)
--> 797 raise AttributeError(name)

AttributeError: dict

It is like if within this environment I did not have access to the database which I do ot understand since I have installed rabbitmq and postgresql globally. Both environment points to the same binary:

(sscha) gjoalland@ctimac001:~$ which psql
/usr/bin/psql
(sscha) gjoalland@ctimac001:~$ conda deactivate
(aiida) gjoalland@ctimac001:~$ which psql
/usr/bin/psql

Could you test this feature or help me resolve this issue?

Thanks for your help!

2

Hi @gjoalland13

could you please share which AiiDA version you are using in both cases? Either run verdi status or pip list | grep aiida.

In the environment where you encounter the issue, what is the output of simply running n.outputs.output_parameters or n.outputs.output_parameters.get_dict()?

Best,
Timo

3

Hi @t-reents,
Thanks for the reply. I have installed aiida-core version 2.6.2.post0 by cloning the github repository in both cases:

(aiida) gjoalland@ctimac001:~$ verdi status
 ✔ version:     AiiDA v2.6.2.post0
 ✔ config:      /home/gjoalland/.aiida
 ✔ profile:     gjoalland
 ✔ storage:     Storage for 'gjoalland' [open] @ postgresql+psycopg://gjoalland:***@localhost:5432/aiida_database / DiskObjectStoreRepository: c228ae00760a4523aa0788862923716d | /home/gjoalland/repository/gjoalland/container
 ✔ broker:      RabbitMQ v3.12.1 @ amqp://guest:guest@127.0.0.1:5672?heartbeat=600
 ✔ daemon:      Daemon is running with PID 511294
(aiida) gjoalland@ctimac001:~$ pip list | grep aiida
aiida-core                2.6.2.post0 /home/gjoalland/work/setup/aiida-core
aiida-phonopy             1.2.0
aiida-pseudo              1.7.0       /home/gjoalland/work/setup/aiida-pseudo
aiida-pythonjob           0.1.8
aiida-quantumespresso     4.9.0       /home/gjoalland/work/setup/aiida-quantumespresso
aiida-quantumespresso-ph  0.1.0       /home/gjoalland/work/setup/aiida-quantumespresso-ph
aiida-vibroscopy          1.2.0       /home/gjoalland/work/setup/aiida-vibroscopy
(aiida) gjoalland@ctimac001:~$ conda activate sscha
(sscha) gjoalland@ctimac001:~$ pip list | grep aiida
aiida-core                2.6.2.post0 /home/gjoalland/work/setup/aiida-core
aiida-pseudo              1.7.0       /home/gjoalland/work/setup/aiida-pseudo
aiida-quantumespresso     4.9.0       /home/gjoalland/work/setup/aiida-quantumespresso
(sscha) gjoalland@ctimac001:~$ verdi status
 ✔ version:     AiiDA v2.6.2.post0
 ✔ config:      /home/gjoalland/.aiida
 ✔ profile:     gjoalland
 ✔ storage:     Storage for 'gjoalland' [open] @ postgresql+psycopg://gjoalland:***@localhost:5432/aiida_database / DiskObjectStoreRepository: c228ae00760a4523aa0788862923716d | /home/gjoalland/repository/gjoalland/container
 ✔ broker:      RabbitMQ v3.12.1 @ amqp://guest:guest@127.0.0.1:5672?heartbeat=600
 ✔ daemon:      Daemon is running with PID 511294

I really encounter the issue when interacting with the “content” of the node:

(sscha) gjoalland@ctimac001:~$ verdi shell
Python 3.9.21 | packaged by conda-forge | (main, Dec  5 2024, 13:51:40) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.18.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: n=load_node(19376) #node created within the sscha env

In [2]: n
Out[2]: <WorkChainNode: uuid: df719e2e-3176-4163-8f0c-79a3c7c1f528 (pk: 19376) (aiida.workflows:quantumespresso.pw.base)>

In [3]: n.outputs.output_parameters
Out[3]: <Data: uuid: 8d8f3cfe-a75a-4ec7-b3bd-0de90e9b375d (pk: 19784)>

In [4]: n.outputs.output_parameters.get_dict()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[4], line 1
----> 1 n.outputs.output_parameters.get_dict()

File ~/work/setup/aiida-core/src/aiida/orm/nodes/node.py:797, in Node.__getattr__(self, name)
    792     warn_deprecation(
    793         f'`{kls}.{name}` is deprecated, use `{kls}.base.links.{new_name}` instead.', version=3, stacklevel=3
    794     )
    795     return getattr(self.base.links, new_name)
--> 797 raise AttributeError(name)

AttributeError: get_dict
In [5]: n=load_node(19376) #node created within the aiida env

In [6]: n.outputs.output_parameters.get_dict()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[6], line 1
----> 1 n.outputs.output_parameters.get_dict()

File ~/work/setup/aiida-core/src/aiida/orm/nodes/node.py:797, in Node.__getattr__(self, name)
    792     warn_deprecation(
    793         f'`{kls}.{name}` is deprecated, use `{kls}.base.links.{new_name}` instead.', version=3, stacklevel=3
    794     )
    795     return getattr(self.base.links, new_name)
--> 797 raise AttributeError(name)

AttributeError: get_dict

Within the sscha environment, It seems that I can see the database but do not have access to it.

Looking forward for your help,
Gabriel

4

Hi Gabriel,
I think the problem is that you have the same AiiDA config file path

You should change it by exporting the AIIDA_PATH to somewhere else (by default it is $HOME/.aiida. Have a look at this how-to.

This is also particularly useful if you have multiple storage partitions on your workstation, as this is also the path where the aiida storage is placed, meaning where all the “heavy” files are stored. Hence, if you have a large SSD on your computer, think about using a path there.

An other hint. Since exporting the AIIDA_PATH can be tedious, @mbercx also has an aiida-project python package which should help with this. Check it out here. Although, keep in mind that this uses virtual environments instead of conda environments (which may also do the job for you, unless you need specific compilers or python versions among your aiida environments).

5

I’m sorry, but I don’t see why I should change the AIIDA_PATH. In principle, simply creating another profile should be enough to isolate the work I’m doing in both environments. I don’t see why I would need to do this though.

An other hint. Since exporting the AIIDA_PATH can be tedious, @mbercx also has an aiida-project python package which should help with this. Check it out here. Although, keep in mind that this uses virtual environments instead of conda environments (which may also do the job for you, unless you need specific compilers or python versions among your aiida environments).

I indeed need specific and different compilers in both environment, so unfortunately I cannot use the aiida-project package…

Anyway, creating a new instance with a fresh posgresql database and a new profile eventually worked. I created the two instances manually and specified the AIIDA_PATH in the conda startup file. I am still not sure what was wrong though… In the past, I’ve used different conda environments for different projects and never had to create multiple AiiDA instances nor profiles.

Thank you!

6

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.