Error archive import

General Usage
1

Dear all,

I have been using aiida for a while now, and I wanted to import some calculations Data for "Tuning Structural and Electronic Properties of Metal-Organic Framework 5 by Metal Substitution and Linker Functionalization" which I did a while ago. While importing the data, with verdi -p MOF5_original archive import -v Zn_MOF_workflows.aiida, I encountered a critical message:

Critical: an exception occurred while trying to import the archive Zn_MOF_workflows.aiida: UnboundLocalError: cannot access local variable 'handle' where it is not associated with a value

Well, I initialized the profile MOF5_original with verdi presto --use-postgres --profile-name MOF5_original, with aiida-core 2.7.1. I am not sure which version I used to perform the calculations, but I am also not sure if this could cause the error. [Could it have been aiida-core 1.6?]

If you need additional information, please let me know. Any help is appreciated.

All the best

Josh

2

Hi @Scemp

I can reproduce this issue (using v2.7.1 but also using 2.4.3). I commented out the following lines in aiida-core, just to get a full traceback: aiida-core/src/aiida/cmdline/commands/cmd_archive.py at v2.7.1 · aiidateam/aiida-core · GitHub

You can find the full error below. It seems that some of the objects/keys are missing. However, I’m not a 100% what exactly is going on and why this is happening.

Pinging @geiger_j and @agoscinski who worked on the archive lately, if I remember correctly.

Best,
Timo

Traceback (most recent call last):                                                                                                                                                                                    
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/aiida/storage/sqlite_zip/backend.py", line 445, in open
    handle = self._zipfile.open(f'{self._folder}/{key}')
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.12/Frameworks/Python.framework/Versions/3.11/lib/python3.11/zipfile.py", line 1564, in open
    zinfo = self.getinfo(name)
            ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.12/Frameworks/Python.framework/Versions/3.11/lib/python3.11/zipfile.py", line 1493, in getinfo
    raise KeyError(
KeyError: "There is no item named 'repo/526edec725a0f33bc1cc66a48038daec14e21ad47d1d5fc1e081441750b2283f' in the archive"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/aiida/storage/sqlite_zip/backend.py", line 448, in open
    raise FileNotFoundError(f'object with key `{key}` does not exist.')
FileNotFoundError: object with key `526edec725a0f33bc1cc66a48038daec14e21ad47d1d5fc1e081441750b2283f` does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/bin/verdi", line 8, in <module>
    sys.exit(verdi())
             ^^^^^^^
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/aiida/cmdline/groups/verdi.py", line 119, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/aiida/cmdline/utils/decorators.py", line 104, in wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_archive.py", line 411, in import_archive
    _import_archive_and_migrate(ctx, archive, web_based, import_kwargs, migration)
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_archive.py", line 504, in _import_archive_and_migrate
    _import_archive(archive_path, archive_format=archive_format, **import_kwargs)
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/aiida/tools/archive/imports.py", line 204, in import_archive
    _add_files_to_repo(backend_from, backend, new_repo_keys)
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/aiida/tools/archive/imports.py", line 1209, in _add_files_to_repo
    for key, handle in repository_from.iter_object_streams(new_keys):  # type: ignore[arg-type]
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/aiida/storage/sqlite_zip/backend.py", line 383, in iter_object_streams
    with self.open(key) as handle:
  File "/opt/homebrew/Cellar/python@3.11/3.11.12/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/Users/treents/.aiida_venvs/debug-josh-discourse/lib/python3.11/site-packages/aiida/storage/sqlite_zip/backend.py", line 450, in open
    handle.close()
    ^^^^^^
UnboundLocalError: cannot access local variable 'handle' where it is not associated with a value
3

@giovannipizzi

Just summarizing what we discussed and what I tried:

  • I checked and the file mentioned in the error above is indeed the only one that is missing
  • I tried to adjust the archive by adding an empty file, however, this fails because the hash will be different. @Scemp has some issues with the original profile (e.g., storage migration etc.), so we couldn’t reproduce the archive yet
  • Also tried to modify the sqlite DB, i.e., removing the repository_metadata for this key, but AiiDA always fails to read the DB afterwards, even though I can still do it via Python