From bash to AiiDA

Hello, I need a bit of help for getting started with AiiDA. I want to manage a simple workflow which is composed of three steps:

#!/bin/bash

basename="csh_s06_03"

# 1. custom python tool
python3 /home/aiida/cash/scripts/cash_create.py -n 3 3 2 -c 1.2 0.1 1 -w 0.005 -at -f ${basename}.lt -f1 /home/aiida/cash/lt/csh_header.lt -f2 /home/aiida/cash/lt/csh_simulation.lt

# 2. Render with Moltemplate
moltemplate.sh ${basename}.lt

# 3. Run the MD simulation with LAMMPS
mpirun -np 4 lmp__22Jul2 -in ${basename}.in -l ${basename}.log

I am using AiiDA version 2.8.0 in a docker container (aiidateam/aiida-core-with-services:latest) with persistent storage. I set up the following computers:

verdi computer list
(base) aiida@916a9ff68b01:~$ verdi computer show localhost
---------------------------  ------------------------------------
Label                        localhost
PK                           1
UUID                         e66ac2f8-ad3c-407b-b94e-9e4837ae1e6d
Description                  container computer
Hostname                     localhost
Transport type               core.local
Scheduler type               core.direct
Work directory               /home/aiida/aiida_run/
Shebang                      #!/bin/bash
Mpirun command               mpirun -np {tot_num_mpiprocs}
Default #procs/machine       4
Default memory (kB)/machine
Prepend text
Append text
---------------------------  ------------------------------------
(base) aiida@916a9ff68b01:~$ verdi computer show caska
---------------------------  ------------------------------------
Label                        caska
PK                           3
UUID                         dfe74fac-7163-45f9-9c2a-ef763ab6db89
Description                  Otello's laptop
Hostname                     caska
Transport type               core.ssh
Scheduler type               core.direct
Work directory               /home/otello/material_solutions/science/aiida
Shebang                      #!/bin/bash
Mpirun command               mpirun -np {tot_num_mpiprocs}
Default #procs/machine       4
Default memory (kB)/machine
Prepend text
Append text
---------------------------  ------------------------------------

and registered the following codes:

verdi code list
(base) aiida@916a9ff68b01:~$ verdi code show 1
--------------------------  ------------------------------------
PK                          1
UUID                        c07cb6af-789a-403e-b4d8-1a3a8a40c8fa
Type                        core.code.installed
Label                       lmp_22Jul25
Description                 Local copy of LAMMPS
Computer                    caska (caska), pk: 3
Default `CalcJob` plugin    aiida-lammps
Escape using double quotes  False
Run with MPI
Prepend script
Append script
Filepath executable         /usr/local/bin/lmp_22Jul25_update4
--------------------------  ------------------------------------
(base) aiida@916a9ff68b01:~$ verdi code show 2
--------------------------  ------------------------------------
PK                          2
UUID                        7e0d2e14-f7f2-47ac-9436-c454cc37f0db
Type                        core.code.installed
Label                       moltemplate
Description                 Moltemplate on caska
Computer                    caska (caska), pk: 3
Default `CalcJob` plugin    aiida-shell
Escape using double quotes  False
Run with MPI
Prepend script              export PATH="$PATH:/home/otello/material_solutions/software/moltemplate/moltemplate:/home/otello/material_solutions/software/moltemplate/moltemplate/scripts"
Append script
Filepath executable         /home/otello/material_solutions/software/moltemplate/moltemplate/scripts/moltemplate.sh
--------------------------  ------------------------------------
(base) aiida@916a9ff68b01:~$ verdi code show 3
--------------------------  ------------------------------------
PK                          3
UUID                        fd989c0a-c650-40d2-84db-f77ec2bbaf00
Type                        core.code.portable
Label                       cash_create
Description                 tools to create C(A)SH structures with arbitrary composition
Default `CalcJob` plugin    core.shell
Escape using double quotes  False
Run with MPI
Prepend script
Append script
Filepath executable         cash_create.py
--------------------------  ------------------------------------

Now, I started writing an AiiDA script based on aiida-shell implementing the first step, but it fails with the following error:

aiida.common.exceptions.NotExistent: No computer has been set for this calculation

Here is my script:

workflow01.py
from aiida.orm import Float, Int, Str, SinglefileData, load_computer, load_code
from aiida_shell import launch_shell_job
from aiida import load_profile

load_profile()

code_cash_create = load_code('cash_create')

basename = 'csh_s06_03'

results, node = launch_shell_job(
    code_cash_create,
    arguments = (
        '-n {nx} {ny} {nz}'
        ' -c {ca_si_ratio} {al_si_ratio} {water_content}'
        ' -w {tolerance}'
        ' -at'
        ' -f {basename}.lt'
        ' -f1 {file_header}'
        ' -f2 {file_run}'
    ),
    nodes = {
        'nx': Int(3),
        'ny': Int(3),
        'nz': Int(2),
        'ca_si_ratio': Float(1.2),
        'al_si_ratio': Float(0.1),
        'water_content': Float(1),
        'tolerance': Float(0.005),
        'basename': Str(basename),
        'file_header': SinglefileData(file='/home/aiida/cash/lt/csh_header.lt'),
        'file_run': SinglefileData(file='/home/aiida/cash/lt/csh_equilibration.lt'),
    },
    outputs = [f'{basename}.lt'],
    metadata = {
        'computer': load_computer('localhost'),
        'options': {
            'resources': {'num_machines': 1, 'num_mpiprocs_per_machine': 1},
        }
    }
)

To debug my profile setup, I run the following script:

test_date.py
from aiida.orm import load_computer
from aiida_shell import launch_shell_job
from aiida import load_profile

load_profile()
results, node = launch_shell_job(
    'date',
    metadata={'computer': load_computer('caska')}
)
print(results['stdout'].get_content())

And it works, although with a massive overhead! I appreciate any comment on the main workflow.

Otello

Hi @hothello

The issue is that you are using a PortableCode, which seems to have a bug in aiida-shell. I just opened a PR there (Fix `prepare_shell_job_inputs` to be compatible with `PortableCode` by t-reents · Pull Request #119 · aiidateam/aiida-shell · GitHub) and let you know once it’s fixed.

I could reproduce your error and the PR fixed it.

Best,
Timo

Hi Timo,

Thanks for confirming the bug. Meanwhile, I resorted to call the python script from a bash one. A silly workaround, but it allowed me to set up the workflow I wanted.

I guess another approach would have been to import functions in the file and define my own @calcfunction replacing the CLI to the script.

Best wishes,
Otello

Hi Otello,

just to let you know, the PR was just merged: Fix `prepare_shell_job_inputs` to be compatible with `PortableCode` by t-reents · Pull Request #119 · aiidateam/aiida-shell · GitHub

Hence, if you checkout the main branch, you should even be able to use your original version.

Hope that helps and let us know in case you encounter further issues in the future!

Timo