Configure Aiida to use remote HPC

Thank you @sphuber for sharing the links
I tried the API method of running the code and seems like it worked, no errors found. Ran it twice just to reconfirm.

However, we are trying to remove multiple code labels but it seems unrecognizable while we execute the commands…

(aiida) x0144578@dcalph078:~$ verdi code list
Full label             Pk  Entry point
-------------------  ----  -------------------
qe-6.8-pw@uhpc-test     2  core.code.installed
pw@uhpc-test            3  core.code.installed
pw@uhpc-test            4  core.code.installed

Use `verdi code show IDENTIFIER` to see details for a code
2023-11-23 02:17:37.635 PST [46964] LOG:  unexpected EOF on client connection with an open transaction
(aiida) x0144578@dcalph078:~$ verdi code delete pw@uhpc-test
Usage: verdi code delete [OPTIONS] [CODES]...
Try 'verdi code delete --help' for help.

Error: Invalid value for '[CODES]...': multiple Code entries found with LABEL<pw@uhpc-test>
2023-11-23 02:17:57.179 PST [47003] LOG:  unexpected EOF on client connection with an open transaction

Are we missing any points here?

Regards
Shraddha

Hi @Shraddha_Kiran,

as the error indicates, there are multiple codes with the same pw@uhpc-test label, since you have run the command twice. AiiDA currently allows two different codes which have the same label.

Fortunately, you can still use the PK to select the code you want to delete, e.g.

verdi code delete 4

in case you want to delete the last code you have set up here.

Hi @mbercx
Thanks. It’s work on this " verdi code delete 5". Now, we have one code created with following information
aiida) x0144578@dcalph078:~$ verdi code list
Full label Pk Entry point


qe-6.8-pw@uhpc-test 2 core.code.installed

(aiida) x0144578@dcalph078:~$ verdi code show qe-6.8-pw


PK 2
UUID 81bf698b-e309-4f94-b14e-50700e631b13
Type core.code.installed
Computer uhpc-test (dcalph000), pk: 2
Filepath executable pw.x
Label qe-6.8-pw
Description quantum_espresso v6.8
Default calc job plugin quantumespresso.pw
Use double quotes True
With mpi
Prepend text module load app_env/quantum-espresso-6.8
Append text


2023-11-22 09:08:39.961 PST [3990] LOG: unexpected EOF on client connection with an open transaction

After this above process, what is the next setup procedure to submit calculation on remote hpc machine and check the job status for aiida-quantumespresso.

As mentioned in the documentation you can launch a test calculation as follows:

aiida-quantumespresso calculation launch pw -X qe-6.8-pw@uhpc-test -F SSSP/1.2/PBE/efficiency

This will launch a simple SCF calculation for a crystalline silicon structure.

Hi @sphuber
we got this error. we have our own quantum espresso installed in remote HPC. is there way to run the calculation without using pseudo-family installation?

(aiida) x0144578@dcalph078:~$ aiida-pseudo install sssp -x PBEsol

Report: downloading patch versions information… [FAILED]
Critical: (‘Connection aborted.’, ConnectionResetError(104, ‘Connection reset by peer’))
2023-11-22 09:26:03.280 PST [5741] LOG: unexpected EOF on client connection with an open transaction
(aiida) x0144578@dcalph078:~$ aiida-quantumespresso 2023-11-22 10:37:14.695 PST [12928] LOG: unexpected EOF on client connection with an open transaction

Unfortunately without pseudopotentials it will be pretty tricky to run a QE calculation.

It seems that your cluster probably has a firewall that doesn’t allow connections to the internet. Luckily, we have built a solution for this situation in aiida-pseudo.
Essentially what you will have to do is install aiida-pseudo on a machine that has internet access and then run

aiida-pseudo install sssp -x PBEsol --download-only

This will download the family to a file.
You then copy this file over to your cluster and there you execute

aiida-pseudo install sssp --from-download <FILENAME>

replacing <FILENAME> with the name of the file you copied over.

Please provide if any readymade file available for install.
(aiida) x0144578@dcalph075:~$ aiida-pseudo install sssp -x PBEsol --download-only
Critical: Could not connect to database: (psycopg2.OperationalError) connection to server at “localhost” (127.0.0.1), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?

(Background on this error at: Error Messages — SQLAlchemy 1.4 Documentation)

Hi @Arun,

It seems you are running with --download-only on the same machine which has not access to the internet. Could you try to run it on another machine for instance your laptop which has access to the Internet. Then you can get a file which you can upload to your remote cluster by scp or ftp service.

Hi,
I can able to submit the job to remote HPC machine via aiidav.2.4.0 but job failed to run completely.but we can able to run same input directly HPC machine and work well with expected output.
(aiida) x0144578@dcalph078:~$ verdi code show 61


PK 61
UUID 0c773ba6-9491-491e-ac8c-762a83b6076c
Type core.code.installed
Computer uhpc-test (dcalph000), pk: 2
Filepath executable /sw/espresso/qe-6.8/bin/pw.x
Label qe-6.8
Description quantum_espresso v6.8
Default calc job plugin quantumespresso.pw
Use double quotes False
With mpi
Prepend text module load app_env/quantum-espresso-6.5_parallel
Append text


2024-01-08 16:33:42.990 PST [7241] LOG: unexpected EOF on client connection with an open transaction

(aiida) x0144578@dcalph078:~/test_aiida$ verdi code create core.code.installed --config …/code.yml
Report: enter ? for help.
Report: enter ! to ignore the default and set no value.
Escape using double quotes [y/N]: N
Success: Created InstalledCode<100>
2024-01-08 17:18:02.197 PST [14990] LOG: unexpected EOF on client connection with an open transaction
(aiida) x0144578@dcalph078:~/test_aiida$ verdi run scf_calcjob.py
Submitted CalcJob with PK=103
2024-01-08 17:18:13.052 PST [15096] LOG: unexpected EOF on client connection with an open transaction
(aiida) x0144578@dcalph078:~/test_aiida$ 2024-01-08 17:22:41.159 PST [20264] LOG: checkpoint starting: time
2024-01-08 17:22:48.811 PST [20264] LOG: checkpoint complete: wrote 78 buffers (0.5%); 0 WAL file(s) added, 0 removed, 0 recycled; write=7.641 s, sync=0.002 s, total=7.652 s; sync files=48, longest=0.001 s, average=0.001 s; distance=465 kB, estimate=495 kB

(aiida) x0144578@dcalph078:~/test_aiida$ verdi process report 103
*** 103 [c-Si SCF_NC]: None
*** (empty scheduler output file)
*** Scheduler errors:
Loading app_env/quantum-espresso-6.5_parallel
Loading requirement: app_env/intel-ps-2016
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN__ 58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 __libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown

*** 4 LOG MESSAGES:
±> WARNING at 2024-01-08 17:19:48.128068-08:00
| key ‘symmetries’ is not present in raw output dictionary
±> ERROR at 2024-01-08 17:19:48.164194-08:00
| ERROR_OUTPUT_STDOUT_INCOMPLETE
±> ERROR at 2024-01-08 17:19:48.169952-08:00
| Both the stdout and XML output files could not be read or parsed.
±> WARNING at 2024-01-08 17:19:48.172281-08:00
| output parser returned exit code<305>: Both the stdout and XML output files could not be read or parsed.
2024-01-09 10:14:32.767 PST [52295] LOG: unexpected EOF on client connection with an open transaction

Hi,
I can able to submit the job to remote HPC machine via aiidav.2.4.0 but job failed to run completely.but we can able to run same input directly HPC machine and work well with expected output.
(aiida) x0144578@dcalph078:~$ verdi code show 61


PK 61
UUID 0c773ba6-9491-491e-ac8c-762a83b6076c
Type core.code.installed
Computer uhpc-test (dcalph000), pk: 2
Filepath executable /sw/espresso/qe-6.8/bin/pw.x
Label qe-6.8
Description quantum_espresso v6.8
Default calc job plugin quantumespresso.pw
Use double quotes False
With mpi
Prepend text module load app_env/quantum-espresso-6.5_parallel
Append text


2024-01-08 16:33:42.990 PST [7241] LOG: unexpected EOF on client connection with an open transaction

(aiida) x0144578@dcalph078:~/test_aiida$ verdi code create core.code.installed --config …/code.yml
Report: enter ? for help.
Report: enter ! to ignore the default and set no value.
Escape using double quotes [y/N]: N
Success: Created InstalledCode<100>
2024-01-08 17:18:02.197 PST [14990] LOG: unexpected EOF on client connection with an open transaction
(aiida) x0144578@dcalph078:~/test_aiida$ verdi run scf_calcjob.py
Submitted CalcJob with PK=103
2024-01-08 17:18:13.052 PST [15096] LOG: unexpected EOF on client connection with an open transaction
(aiida) x0144578@dcalph078:~/test_aiida$ 2024-01-08 17:22:41.159 PST [20264] LOG: checkpoint starting: time
2024-01-08 17:22:48.811 PST [20264] LOG: checkpoint complete: wrote 78 buffers (0.5%); 0 WAL file(s) added, 0 removed, 0 recycled; write=7.641 s, sync=0.002 s, total=7.652 s; sync files=48, longest=0.001 s, average=0.001 s; distance=465 kB, estimate=495 kB

(aiida) x0144578@dcalph078:~/test_aiida$ verdi process report 103
*** 103 [c-Si SCF_NC]: None
*** (empty scheduler output file)
*** Scheduler errors:
Loading app_env/quantum-espresso-6.5_parallel
Loading requirement: app_env/intel-ps-2016
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN__ 58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pw.x 0000000000FE851D Unknown Unknown Unknown
libpthread-2.17.s 00001555517065D0 Unknown Unknown Unknown
libmpi.so.12 0000155551F47E99 I_MPI_PrintAllEnv Unknown Unknown
libmpi.so.12 0000155551F838AF MPIR_Init_thread Unknown Unknown
libmpi.so.12 0000155551F74890 MPI_Init Unknown Unknown
libmpifort.so.12. 00001555525AB0A0 MPI_INIT Unknown Unknown
pw.x 00000000009256E8 mp_world_mp_mp_wo 78 mp_world.f90
pw.x 0000000000925196 mp_global_mp_mp_s 71 mp_global.f90
pw.x 0000000000406F32 MAIN
58 pwscf.f90
pw.x 0000000000406EDE Unknown Unknown Unknown
libc-2.17.so 000015555104A3D5 __libc_start_main Unknown Unknown
pw.x 0000000000406DE9 Unknown Unknown Unknown

*** 4 LOG MESSAGES:
±> WARNING at 2024-01-08 17:19:48.128068-08:00
| key ‘symmetries’ is not present in raw output dictionary
±> ERROR at 2024-01-08 17:19:48.164194-08:00
| ERROR_OUTPUT_STDOUT_INCOMPLETE
±> ERROR at 2024-01-08 17:19:48.169952-08:00
| Both the stdout and XML output files could not be read or parsed.
±> WARNING at 2024-01-08 17:19:48.172281-08:00
| output parser returned exit code<305>: Both the stdout and XML output files could not be read or parsed.
2024-01-09 10:14:32.767 PST [52295] LOG: unexpected EOF on client connection with an open transaction
[/quote]

Hello @sphuber
We’ve got the necessary aiida pseudo potentials package from offline downloads but we are still unable to run the calculations using aiida singularity image configured with quantumespresso 6.5.
However, when we try to run the same calculation using quantumespresso 6.5 directly on our HPC, they run successfully.
Are we missing any config steps in aiida image? Please guide.

Thank you
Shraddha

we have download the package and try to install the package. we can’t able to proceed this step:
aiida-pseudo install sssp --from-download

we are getting the below error
aiida) x0144578@dcalph078:~$ aiida-pseudo install sssp --from-download test_aiida/archive.tar.gz
Usage: aiida-pseudo install sssp [OPTIONS]
Try ‘aiida-pseudo install sssp -h’ for help.

Error: No such option: --from-download Did you mean --download-only?
(aiida) x0144578@dcalph078:~$ aiida-pseudo install sssp -h
Usage: aiida-pseudo install sssp [OPTIONS]

Install an SSSP configuration.

The SSSP configuration will be automatically downloaded from the Materials
Cloud Archive entry to create a new SsspFamily.

(aiida) x0144578@dcalph078:~ aiida-pseudo install family test_aiida/archive.tar.gz sssp -f gztar Report: unpacking archive and parsing pseudos... [FAILED] Critical: `<class 'aiida_pseudo.data.pseudo.pseudo.PseudoPotentialData'>` constructor did not define the element and could not parse a valid element symbol from the filename `{filename}` either. It should have the format `ELEMENT.EXTENSION` 2024-01-16 06:48:47.291 PST [25023] LOG: unexpected EOF on client connection with an open transaction (aiida) x0144578@dcalph078:~ aiida-pseudo install family test_aiida/archive.tar.gz sssp
Report: unpacking archive and parsing pseudos… [FAILED]
Critical: <class 'aiida_pseudo.data.pseudo.pseudo.PseudoPotentialData'> constructor did not define the element and could not parse a valid element symbol from the filename {filename} either. It should have the format ELEMENT.EXTENSION
2024-01-16 06:51:11.920 PST [25403] LOG: unexpected EOF on client connection with an open transaction
(aiida) x0144578@dcalph078:~$ aiida-pseudo install family test_aiida/archive.tar.gz sssp -F pseudo.family.sssp
Usage: aiida-pseudo install family [OPTIONS] ARCHIVE LABEL
Try ‘aiida-pseudo install family -h’ for help.

Error: Invalid value for ‘-F’ / ‘–family-type’: pseudo.family.sssp is not an accepted value for this option.

(aiida) x0144578@dcalph078:~$ verdi plugin list aiida.groups | grep pseudo.family

  • pseudo.family
  • pseudo.family.cutoffs
  • pseudo.family.pseudo_dojo
  • pseudo.family.sssp

Please update aiida-pseudo with pip install -U aiida-pseudo and then try again

Hi sphuber,
I am getting an error, file is not found.
(aiida) x0144578@dcalph078:~$ aiida-pseudo install sssp --from-download test_aiida/archive.tar.gz
Critical: archive test_aiida/archive.tar.gz did not contain required file archive.tar.gz

(aiida) x0144578@dcalph078:~$ cd test_aiida/
archive.tar.gz metadata.json scf_calcjob.py Si.cif Si.pbe-n-kjpaw_psl.1.0.0.UPF

(aiida) x0144578@dcalph078:~$ cd test_aiida/

That is not the correct archive. You should run the following on a machine with access to the internet:

$ aiida-pseudo install sssp --download-only
Report: downloading patch versions information...  [OK]
Report: downloading selected pseudopotentials archive...  [OK]
Report: downloading selected pseudopotentials metadata...  [OK]
Success: Pseudopotential archive written to: SSSP_1.3_PBE_efficiency.aiida_pseudo

As you can see, that creates the file SSSP_1.3_PBE_efficiency.aiida_pseudo. You transfer this to the machine where you want to run AiiDA and you run

aiida-pseudo install sssp --from-download SSSP_1.3_PBE_efficiency.aiida_pseudo

I am following this below step. Please guide me to run a test 2. we facing issue in upload sssp archieve and change a script line accordingly. Please provide command for this. The above command we cant able to perform.

(aiida) x0144578@dcalph075:~/test_aiida$ aiida-pseudo install sssp --download-only
Report: downloading patch versions information… [FAILED]
Critical: (‘Connection aborted.’, ConnectionResetError(104, ‘Connection reset by peer’))
(aiida) x0144578@dcalph075:~/test_aiida$ ls -ltr
total 99040
-rwx------ 1 x0144578 hpcusers 1691876 Jan 3 01:45 Si.pbe-n-kjpaw_psl.1.0.0.UPF
-rwx------ 1 x0144578 hpcusers 746 Jan 3 01:55 Si.cif
-rwxr-xr-x 1 x0144578 hpcusers 2360 Jan 8 07:06 scf_calcjob.py
-rwxr-xr-x 1 x0144578 hpcusers 18114 Jan 16 05:42 metadata.json
-rwxr-xr-x 1 x0144578 hpcusers 38814119 Jan 16 05:43 archive.tar.gz
-rwx------ 1 x0144578 hpcusers 60464715 Jan 17 12:53 SSSP_1.3.0_PBEsol_efficiency.tar.gz
(aiida) x0144578@dcalph078:~$ aiida-pseudo install sssp --from-download archive.tar.gz
Critical: archive archive.tar.gz did not contain required file archive.tar.gz

we are getting fail to upload this tar file.we can able to upload as below way.

(aiida) x0144578@dcalph075:~/test_aiida$ aiida-pseudo install family archive.tar.gz sssp -P pseudo.upf
Report: unpacking archive and parsing pseudos… [OK]
Success: installed sssp containing 85 pseudopotentials
2024-01-18 07:20:42.374 PST [42388] LOG: unexpected EOF on client connection with an open transaction

(aiida) x0144578@dcalph075:~/test_aiida$ verdi plugin list aiida.data | grep pseudo

  • pseudo
  • pseudo.jthxml
  • pseudo.psf
  • pseudo.psml
  • pseudo.psp8
  • pseudo.upf
  • pseudo.vps

(aiida) x0144578@dcalph075:~/test_aiida$ verdi plugin list aiida.groups | grep pseudo.family

  • pseudo.family
  • pseudo.family.cutoffs
  • pseudo.family.pseudo_dojo
  • pseudo.family.sssp

Test1: By uploading upf file using command:verdi data core.upf import, we can run sample test.

Select structure

structure=orm.load_node(8)
scf_builder.structure=structure

Select pseudopotentials

##pseudos={‘Si’ : orm.load_node(1997)}
pseudos={‘Si’ : orm.load_node(7)}
scf_builder.pseudos=pseudos

Test2: we are failing with error for this workchain sample run.

Select structure

structure=orm.load_node(250)
#relax_builder.structure=structure

Select pseudopotentials

pseudo_family = orm.load_group(‘sssp’)
pseudos = pseudo_family.get_pseudos(structure=structure)
#relax_builder.pseudos=pseudos

(aiida) x0144578@dcalph075:~/test_aiida$ verdi run workchain.py
Traceback (most recent call last):
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida/orm/utils/loaders.py”, line 407, in load_entity
entity = builder.one()[0]
^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida/orm/querybuilder.py”, line 1129, in one
raise NotExistent(‘No result was found’)
aiida.common.exceptions.NotExistent: No result was found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/user/x0144578/.conda/envs/aiida/bin/verdi”, line 10, in
sys.exit(verdi())
^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/utils/decorators.py”, line 73, in wrapper
return wrapped(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_run.py”, line 115, in run
exec(compile(handle.read(), str(filepath), ‘exec’, dont_inherit=True), globals_dict) # pylint: disable=exec-used
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “workchain.py”, line 19, in
pseudo_family = orm.load_group(‘SSSP/1.1/PBE/efficiency’)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida/orm/utils/loaders.py”, line 177, in load_group
return load_entity(
^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida/orm/utils/loaders.py”, line 88, in load_entity
return entity_loader.load_entity(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida/orm/utils/loaders.py”, line 413, in load_entity
raise NotExistent(error)
aiida.common.exceptions.NotExistent: no Group found with LABEL<SSSP/1.1/PBE/efficiency>: No result was found
2024-01-23 13:03:17.878 PST [22561] LOG: unexpected EOF on client connection with an open transaction
(aiida) x0144578@dcalph075:~/test_aiida$ vi scf_calcjob.py
(aiida) x0144578@dcalph075:~/test_aiida$ verdi run workchain.py
Traceback (most recent call last):
File “/user/x0144578/.conda/envs/aiida/bin/verdi”, line 10, in
sys.exit(verdi())
^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/utils/decorators.py”, line 73, in wrapper
return wrapped(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_run.py”, line 115, in run
exec(compile(handle.read(), str(filepath), ‘exec’, dont_inherit=True), globals_dict) # pylint: disable=exec-used
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “workchain.py”, line 23, in
scf_builder.pseudos=pseudos
^^^^^^^^^^^
NameError: name ‘scf_builder’ is not defined
2024-01-23 13:05:43.885 PST [23136] LOG: unexpected EOF on client connection with an open transaction
(aiida) x0144578@dcalph075:~/test_aiida$ vi scf_calcjob.py
(aiida) x0144578@dcalph075:~/test_aiida$ vi workchain.py
(aiida) x0144578@dcalph075:~/test_aiida$ vi scf_calcjob.py
(aiida) x0144578@dcalph075:~/test_aiida$ verdi run workchain.py
Traceback (most recent call last):
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida_quantumespresso/workflows/pw/base.py”, line 161, in get_builder_from_protocol
pseudo_family = orm.QueryBuilder().append(pseudo_set, filters={‘label’: pseudo_family}).one()[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida/orm/querybuilder.py”, line 1129, in one
raise NotExistent(‘No result was found’)
aiida.common.exceptions.NotExistent: No result was found

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “/user/x0144578/.conda/envs/aiida/bin/verdi”, line 10, in
sys.exit(verdi())
^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/click/core.py”, line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/utils/decorators.py”, line 73, in wrapper
return wrapped(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_run.py”, line 115, in run
exec(compile(handle.read(), str(filepath), ‘exec’, dont_inherit=True), globals_dict) # pylint: disable=exec-used
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “workchain.py”, line 100, in
bands_builder = PwBandsWorkChain.get_builder_from_protocol(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida_quantumespresso/workflows/pw/bands.py”, line 139, in get_builder_from_protocol
relax = PwRelaxWorkChain.get_builder_from_protocol(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida_quantumespresso/workflows/pw/relax.py”, line 109, in get_builder_from_protocol
base = PwBaseWorkChain.get_builder_from_protocol(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/user/x0144578/.conda/envs/aiida/lib/python3.11/site-packages/aiida_quantumespresso/workflows/pw/base.py”, line 163, in get_builder_from_protocol
raise ValueError(
ValueError: required pseudo family SSSP/1.2/PBEsol/efficiency is not installed. Please use aiida-pseudo install toinstall it.
2024-01-23 13:11:07.202 PST [24493] LOG: unexpected EOF on client connection with an open transaction

Please guide or suggest some way to load SSSP/1.2/PBEsol/efficiency file.So we can run the test2.

Hello @sphuber
We tried installing the archive from the machine having internet access which gave us
(aiida):~$ aiida-pseudo install sssp -v 1.1 --download-only
Report: downloading patch versions information… [OK]
Report: downloading selected pseudopotentials archive… [OK]
Report: downloading selected pseudopotentials metadata… [OK]
Success: archive.tar.gz written to the current directory.
Success: metadata.json written to the current directory.

This is slightly different from what we see in your terminal. However, we tried to download the specific archive SSSP_1.3_PBE_efficiency.aiida_pseudo from the internet ( Materials Cloud Archive) and transferred to the machine where we plan to run aiida.

The challenge now is that we are unable to load/install the complete archive folder (which would have multiple .UPF files) but rather load the .UPF files one by one. Is it possible to have the complete archive folder uploaded at once so that our aiida workflows can detect the required material UPF files from the archive automatically, as per the code?
Currently we are able to load single .UPF files using

aiida-pseudo install family archive.tar.gz sssp -P pseudo.upf
Report: unpacking archive and parsing pseudos… [OK]
Success: installed sssp containing 85 pseudopotentials

Thanks
Shraddha

The difference is just the version number of SSSP. If you want v1.3, change the command to:

aiida-pseudo install sssp -v 1.3 --download-only

The archive that is downloaded this way is different from the archives that are on the Materials Cloud. They are not the same. The archives from Materials Cloud are not compatible with aiida-pseudo install sssp --from-download. You should only use the archive that was downloaded with the same command.

If you need SSSP/1.2/PBEsol/efficiency you need to specify v1.2 and PBEsol as the functional when downloading the archive:

aiida-pseudo install sssp -v 1.2 -x PBEsol --download-only

Thank you for the correction. I have another question,
The challenge now is that we are unable to load/install the complete archive folder (which would have multiple .UPF files) but rather load the .UPF files one by one. Is it possible to have the complete archive folder uploaded at once so that our aiida workflows can detect the required material UPF files from the archive automatically, as per the code?
Currently we are able to load single .UPF files using
aiida-pseudo install family archive.tar.gz sssp -P pseudo.upf
Report: unpacking archive and parsing pseudos… [OK]
Success: installed sssp containing 85 pseudopotentials

Thank you
Shraddha

Thank you @sphuber and everyone for helping us out. Your suggestions were really helpful and we were able to run our simulations successfully, finally!

Thank you again
Regards
Shraddha

1 Like