Aiida vibroscopy: Problem in running the code successfully

Hi all,

I am new to AiiDA in general and I want to calculate Raman from aiida-vibroscopy.
I prepared following overrides.yaml and files (please see below). When I check the status, I feel like it wasnt running successfully. Can you please suggest me what I am doing wrong here and how may I fix this?

The was successfully submitted with PK=42
I am not sure what is wrong there but something is not right. Can you please tell me whats going on here?

Then when I check the status:

(aiidaENV) rkarkee@ch-fe1:/lustre/scratch5/rkarkee> verdi process status 42
IRamanSpectraWorkChain<42> Finished [400] [1:inspect_process]
└── HarmonicWorkChain<44> Finished [400] [2:inspect_processes]
├── generate_preprocess_data<45> Finished [0]
├── PhononWorkChain<50> Finished [400] [3:inspect_base_supercell]
│ ├── generate_preprocess_data<51> Finished [0]
│ ├── get_supercell<57> Finished [0]
│ └── PwBaseWorkChain<60> Finished [301] [2:while_(should_run_process)(2:inspect_process)]
│ └── PwCalculation<63> Excepted
└── DielectricWorkChain<55> Excepted [2:run_base_scf]

And Process report 44 shows:

(aiidaENV) rkarkee@ch-fe1:/lustre/scratch5/rkarkee> verdi process report 44
2024-02-14 20:49:59 [5 | REPORT]: [44|HarmonicWorkChain|run_phonon]: submitting `PhononWorkChain` <PK=50>
2024-02-14 20:49:59 [6 | REPORT]: [44|HarmonicWorkChain|run_dielectric]: submitting `DielectricWorkChain` <PK=55>
2024-02-14 20:50:00 [7 | REPORT]: [55|DielectricWorkChain|on_except]: Traceback (most recent call last):
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/engine/”, line 1969, in _exec_single_context
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/engine/”, line 922, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/plumpy/base/”, line 324, in transition_to
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/plumpy/base/”, line 388, in _enter_next_state
self._fire_state_event(StateEventHook.ENTERED_STATE, last_state)
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/plumpy/base/”, line 300, in _fire_state_event
callback(self, hook, state)
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/plumpy/”, line 331, in
lambda _s, <em>h, from_state: self.on_entered(cast(Optional[process_states.State], from_state)),
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/aiida/engine/processes/”, line 424, in on_entered
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/aiida/engine/”, line 287, in set_process_state_change</em> timestamp
backend.set_global_variable(key, value, description)
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/aiida/storage/psql_dos/”, line 411, in set_global_va riable
session.query(DbSetting).filter(DbSetting.key == key).update(dict(val=value))
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/orm/”, line 3271, in update
result: CursorResult[Any] = self.session.execute(
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/orm/”, line 2308, in execute
return self._execute_internal(
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/orm/”, line 2190, in _execute_internal
result: Result[Any] = compile_state_cls.orm_execute_statement(
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/orm/”, line 1617, in orm_execute _statement
return super().orm_execute_statement(
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/orm/”, line 293, in orm_execute_statement
result = conn.execute(
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/engine/”, line 1416, in execute
return meth(
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/sql/”, line 517, in _execute_on_connecti on
return connection._execute_clauseelement(
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/engine/”, line 1639, in _execute_clauseeleme nt
ret = self._execute_context(
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/engine/”, line 1848, in _execute_context
return self._exec_single_context(
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/engine/”, line 1988, in _exec_single_context
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/engine/”, line 2344, in _handle_dbapi_except ion
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/engine/”, line 1969, in _exec_single_context
File “/users/rkarkee/conda/envs/aiidaENV/lib/python3.11/site-packages/sqlalchemy/engine/”, line 922, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
[SQL: UPDATE db_dbsetting SET val=?, time=? WHERE db_dbsetting.“key” = ?]
[parameters: (‘“2024-02-14T13:50:00.839680-07:00”’, ‘2024-02-14 13:50:00.842878’, ‘process|state_change|work’)]
(Background on this error at: [Error Messages — SQLAlchemy 2.0 Documentation](

2024-02-14 20:50:00 [8 | REPORT]: [55|DielectricWorkChain|on_terminated]: remote folders will not be cleaned
2024-02-14 20:50:01 [9 | REPORT]: [50|PhononWorkChain|run_base_supercell]: launching base supercell scf PwBaseWorkChain<60>
2024-02-14 20:50:02 [10 | REPORT]: [60|PwBaseWorkChain|run_process]: launching PwCalculation<63> iteration #1
2024-02-14 20:50:02 [12 | REPORT]: [60|PwBaseWorkChain|on_terminated]: remote folders will not be cleaned
2024-02-14 20:50:02 [13 | REPORT]: [50|PhononWorkChain|inspect_base_supercell]: base supercell scf failed with exit status 301
2024-02-14 20:50:03 [14 | REPORT]: [50|PhononWorkChain|on_terminated]: remote folders will not be cleaned
2024-02-14 20:50:03 [15 | REPORT]: [44|HarmonicWorkChain|inspect_processes]: the child `PhononWorkChain` with <PK=50> failed
2024-02-14 20:50:03 [16 | REPORT]: [44|HarmonicWorkChain|on_terminated]: remote folders will not be cleaned

My inputs for this run is as follow:
I have overrides.yaml file and then a python file (see below)

%%%%%%%%%%%%% overrides.yaml %%%%%%%%%%%%%%%%
clean_workdir: false # whether to clean the working directiories
clean_workdir: false
kpoints_parallel_distance: 0.2 # kpoints distance in Angstrom^-1 to sample the BZ parallel to the electric field. If used, it should help in converging faster the final results
property: raman

# central_difference: # if you know what you are doing, custom numerical derivatives with respect to electric field

# accuracy: 2

# electric_field_step: 0.0005

pseudo_family: new_PBE
kpoints_distance: 0.4 # kpoints distance in Angstrom^-1 to sample the BZ
kpoints_force_parity: false
max_iterations: 5
max_wallclock_seconds: 43200
num_machines: 1
num_mpiprocs_per_machine: 256
#queue_name: standard # for SLURM
# account: account_name # for SLURM, also for project etc
withmpi: true
conv_thr: 2.0e-10
electron_maxstep: 100
mixing_beta: 0.2
ecutrho: 280.0
ecutwfc: 70.0
vdw_corr: Grimme-D2

sleep_submission_time: 1.0
clean_workdir: false
distance: 0.01 # atomic displacements for phonon calculation, in Angstrom
pseudo_family: new_PBE
kpoints_distance: 0.15 # kpoints distance in Angstrom^-1 to sample the BZ
kpoints_force_parity: false
max_iterations: 5
max_wallclock_seconds: 43200
num_machines: 1
num_mpiprocs_per_machine: 1
# queue_name: partition_name # for SLURM
# account: account_name # for SLURM, also for project etc
withmpi: true
cmdline: [‘-nk’, ‘8’]
# gamma_only: True # to use only if KpointsData has only a mesh 1 1 1 0 0 0 (i.e. Gamma not shifted)
conv_thr: 2.0e-12
electron_maxstep: 80
mixing_beta: 0.4

      ecutwfc: 70.0
      ecutrho: 280

      vdw_corr: Grimme-D2

sleep_submission_time: 1.0 # waiting time in seconds between different submission of SCF calculation. Recommended to be at least 1 second, to not overload.
run_parallel: true
use_primitive_cell: false
distinguish_kinds: false
is_symmetry: true
symprec: 1.0e-05

%%%%% %%%%%%%%%%%%%%%%%

# -*- coding: utf-8 -*-

# pylint: disable=line-too-long,wildcard-import,pointless-string-statement,unused-wildcard-import

“”“Submit an IRamanSpectraWorkChain via the get_builder_from_protocol using the overrides.”“”
from pathlib import Path

from aiida import load_profile
from aiida.engine import submit
from aiida.orm import *
from aiida_quantumespresso.common.types import ElectronicType

from aiida_vibroscopy.workflows.spectra.iraman import IRamanSpectraWorkChain


# =============================== INPUTS ===============================

# Please, change the following inputs.

mesh = [[4, 4, 2], [0.5, 0.5, 0.5]]
pw_code_label = ‘qe-7.3@newhpc’
structure_id = 21 # PK or UUID of your AiiDA StructureData
protocol = ‘fast’ # also ‘moderate’ and ‘precise’; ‘moderate’ should be good enough in general
overrides_filepath = ‘./overrides.yaml’ # should be a path, e.g. /path/to/overrides.yaml. Format is YAML

# Consult the documentation for HOW-TO for how to use properly the overrides.

# !!! FOR FULL INPUT NESTED STRUCTURE: [IRamanSpectraWorkChain — aiida-vibroscopy 1.0.2 documentation](

# You can follow the input structure provided on the website to fill further the overrides.

# ======================================================================

# If you don’t have a StructureData, but you have a CIF or XYZ, or similar, file

# you can import your structure uncommenting the following:

# from []( import read

# atoms = read(‘/path/to/file.cif’)

# structure = StructureData(ase=atoms)


# structure_id =

# print(f"Your structure has been stored in the database with PK={structure_id}")

def main():
“”“Submit an IRamanSpectraWorkChain calculation.”“”
code = load_code(pw_code_label)
structure = load_node(structure_id)
kwargs = {‘electronic_type’: ElectronicType.INSULATOR}

kpoints = KpointsData()
kpoints.set_kpoints_mesh(mesh[0], mesh[1])

#pseudo_family = load_group(pseudo_family_name)
#pseudos = pseudo_family.get_pseudos(structure=structure)

builder = IRamanSpectraWorkChain.get_builder_from_protocol(

builder.dielectric.scf.kpoints = kpoints
builder.dielectric.pop('kpoints_parallel_distance', None)
builder.dielectric.scf.pop('kpoints_distance', None)
builder.phonon.scf.kpoints = kpoints = pseudos = pseudos

calc = submit(builder)
print(f'Submitted IRamanSpectraWorkChain with PK={} and UUID={calc.uuid}')
print('Register *at least* the PK number, e.g. in you submit script.')
print('You can monitor the status of your calculation with the following commands:')
print('  * verdi process status PK')
print('  * verdi process list -L IRamanSpectraWorkChain # list all running IRamanSpectraWorkChain')
    '  * verdi process list -ap1 -L IRamanSpectraWorkChain # list all IRamanSpectraWorkChain submitted in the previous 1 day'
print('If the WorkChain finishes with exit code 0, then you can inspect the outputs and post-process the data.')
print('Use the command')
print('  * verdi process show PK')
print('To show further information about your WorkChain. When finished, you should see some outputs.')
print('The main output can be accessed via `load_node(PK).outputs.vibrational_data.numerical_accuracy_*`.')
print('You have to complete the remaning `*`, which depends upond the accuracy of the calculation.')
print('See also the documentation and the reference paper for further details.')

if **name** == ‘**main**’:
“”“Run script.”“”

Hello, as you can see the error comes from SQL, not from aiida-vibroscopy. Unfortunately I wouldn’t know how to help here, so I ping @sphuber who may provide some insights.

This is an error from sqlite. This normally never occurs, but you are now using the core.sqlite_dos backend for your storage (instead of the default core.psql_dos) where postgres is swapped out for sqlite. I recommended this because you had trouble getting Postgres to work. However, this Sqlite based backend is experimental and not suited for production. It looks like this workflow might lead to problems with the database.

Does the same error happen every time you launch the script? Or does it error in different places with different errors?

The real solution is to get Postgres working on your machine and make a normal profile with the default storage backend.

Hi @sphuber

How can I make Postgres working on my machine and use default core.psql_dos for backend storage?
Is there any instructions I can follow to do that?

Those are the default instructions found on the documentation: Getting started — AiiDA 2.5.1.post0 documentation

Hi @sphuber
At the beginning I created a lot of profiles. How do I start from the scrach again? Deleting everything and starting over. I am in HPC system.

(aiidaENV) rkarkee@ch-fe2:/lustre/scratch5/rkarkee> verdi profile list
Report: configuration folder: /users/rkarkee/.aiida

  • new-profile

Since you had trouble with the postgres instance in the conda env, I would just delete everything and start from scratch.
Delete the .aiida folder with rm -rf /users/rkarkee/.aiida and then delete the conda env.

Still, I would not recommend trying to run AiiDA directly on the HPC machine. Run it on your personal machine instead. If that is running Windows, install WSL2 with Ubuntu as image (Install WSL | Microsoft Learn) and then follow these instructions: System-wide installation — AiiDA 2.5.1.post0 documentation

Hi @sphuber

Interestingly, when I created the profile first time in HPC workspace, I had success as seeting up in psotgres. Then I guess I tried with multiple verdi setup where I might have messed up something.

(aiidaENV) rkarkee@ch-fe1:~> verdi status
 ✔ version:     AiiDA v2.5.1
 ✔ config:      /users/rkarkee/.aiida
 ✔ profile:     rkarkee
 ✔ storage:     Storage for 'rkarkee' [open] @ postgresql://aiida_qs_rkarkee_20554bcc4bead70a3479c4ef8d5f1f4e:***@localhost:5432/rkarkee_rkarkee_20554bcc4bead70a3479c4ef8d5f1f4e / DiskObjectStoreRepository: 7ef673c3acf444d2ac6ff9f6c839a15f | /users/rkarkee/.aiida/repository/rkarkee/container
 ✔ rabbitmq:    Connected to RabbitMQ v3.8.14 as amqp://guest:guest@
 ✔ daemon:      Daemon is running with PID 196463
(aiidaENV) rkarkee@ch-fe1:~>

As you suggested, I have installed WSL2 and trying to install aiida there.

While doing verdi quicksetup I ran into this error:

(aiida) rkarkee@RKARKEE:~$ verdi quicksetup
/home/rkarkee/envs/aiida/lib/python3.10/site-packages/aiida/manage/configuration/ UserWarning: Creating AiiDA configuration folder `/home/rkarkee/.aiida`.
  warnings.warn(f'Creating AiiDA configuration folder `{path}`.')
Report: enter ? for help.
Report: enter ! to ignore the default and set no value.
Profile name [quicksetup]: rkee
Email Address (for sharing data):
First name [John]: Rijan
Last name [Doe]: Karkee
Institution [Unknown]:
Trying to become 'postgres' user. You may be asked for your 'sudo' password.
[sudo] password for rkarkee:
Trying to become 'postgres' user. You may be asked for your 'sudo' password.
Trying to become 'postgres' user. You may be asked for your 'sudo' password.
Trying to become 'postgres' user. You may be asked for your 'sudo' password.
Trying to become 'postgres' user. You may be asked for your 'sudo' password.
Trying to become 'postgres' user. You may be asked for your 'sudo' password.
ERROR:  invalid locale name: "en_US.UTF-8"

Error: Oops! quicksetup was unable to create the AiiDA database for you.
            See `verdi quicksetup -h` for how to specify non-standard parameters for the postgresql connection.
            Alternatively, create the AiiDA database yourself:

            Run the following commands as a UNIX user with access to PostgreSQL (Ubuntu: $ sudo su postgres):

        $ psql template1
        ==> CREATE USER "aiida_qs_rkarkee_c67355a95f0f30138e43b517f5e739a9" WITH PASSWORD '<password>'
        ==> GRANT "aiida_qs_rkarkee_c67355a95f0f30138e43b517f5e739a9" TO current_user
        ==> CREATE DATABASE "rkee_rkarkee_c67355a95f0f30138e43b517f5e739a9" OWNER "aiida_qs_rkarkee_c67355a95f0f30138e43b517f5e739a9" ENCODING 'UTF8' LC_COLLATE='en_US.UTF-8' LC_CTYPE='en_US.UTF-8' TEMPLATE=template0

            and then use `verdi setup` instead.

Traceback (most recent call last):
  File "/home/rkarkee/envs/aiida/bin/verdi", line 8, in <module>
  File "/home/rkarkee/envs/aiida/lib/python3.10/site-packages/click/", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/rkarkee/envs/aiida/lib/python3.10/site-packages/click/", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/rkarkee/envs/aiida/lib/python3.10/site-packages/click/", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/rkarkee/envs/aiida/lib/python3.10/site-packages/click/", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/rkarkee/envs/aiida/lib/python3.10/site-packages/click/", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/rkarkee/envs/aiida/lib/python3.10/site-packages/click/", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/rkarkee/envs/aiida/lib/python3.10/site-packages/aiida/cmdline/commands/", line 224, in quicksetup
    raise exception
  File "/home/rkarkee/envs/aiida/lib/python3.10/site-packages/aiida/cmdline/commands/", line 214, in quicksetup
    db_username, db_name = postgres.create_dbuser_db_safe(dbname=db_name, dbuser=db_username, dbpass=db_password)
  File "/home/rkarkee/envs/aiida/lib/python3.10/site-packages/aiida/manage/external/", line 223, in create_dbuser_db_safe
    self.create_db(dbuser=dbuser, dbname=dbname)
  File "/home/rkarkee/envs/aiida/lib/python3.10/site-packages/aiida/manage/external/", line 169, in create_db
    self.execute(_CREATE_DB_COMMAND.format(dbname, dbuser))
  File "/home/rkarkee/envs/aiida/lib/python3.10/site-packages/pgsu/", line 120, in execute
    return _execute_su_psql(command, dsn)
  File "/home/rkarkee/envs/aiida/lib/python3.10/site-packages/pgsu/", line 368, in _execute_su_psql
  File "/usr/lib/python3.10/", line 457, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '['sudo', '-n', 'su', 'postgres', '-c', 'cd ~ && psql -d template1 -p 5432 -tc \'CREATE DATABASE "rkee_rkarkee_c67355a95f0f30138e43b517f5e739a9" OWNER "aiida_qs_rkarkee_c67355a95f0f30138e43b517f5e739a9" ENCODING \'"\'"\'UTF8\'"\'"\' LC_COLLATE=\'"\'"\'en_US.UTF-8\'"\'"\' LC_CTYPE=\'"\'"\'en_US.UTF-8\'"\'"\' TEMPLATE=template0\'']'