# Processing calcjob outputs in BaseRestartWorkChain

**URL:** <https://aiida.discourse.group/t/processing-calcjob-outputs-in-baserestartworkchain/391>\
**Category:** General Usage\
**Tags:** aiida\
**Created:** [May 19, 2024, 2:22am UTC](https://aiida.discourse.group/t/processing-calcjob-outputs-in-baserestartworkchain/391 "2024-05-19T02:22:57Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![cote3804](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/cote3804/32/99_2.png) [@cote3804](https://aiida.discourse.group/u/cote3804)\
**Post date:** [May 19, 2024, 2:22am UTC](https://aiida.discourse.group/t/processing-calcjob-outputs-in-baserestartworkchain/391/1 "2024-05-19T02:22:57Z")

</div>

Hi all,

I’ve started converting my workchains to BaseRestartWorkChains and I’m generally confused about how to grab the outputs from the calcjobs I define in the \_process\_class.

I’m currently using the orca plugin in a work chain to relax a molecule and extract the homo/lumo energy values:

```auto
OrcaCalculation = CalculationFactory("orca.orca")

class HomoLumoWorkChain(BaseRestartWorkChain):

    _process_class = OrcaCalculation

    @classmethod
    def define(cls, spec):
        """ Homo Lumo Calculation WorkChain """
        super().define(spec)
        spec.input('structure', valid_type=StructureData)
        spec.input('code', valid_type=AbstractCode)
        spec.input('parameters', valid_type=Dict)
        spec.input('resources', valid_type=Dict) # keep this a regular dictionary
        spec.outline(
            cls.setup,
            while_(cls.should_run_process)(
                cls.run_process,
                cls.inspect_process,
            ),
            cls.extract_homo_lumo,
            cls.results,
        )
        spec.expose_outputs(OrcaCalculation)
        spec.output('homo', valid_type=Float)
        spec.output('lumo', valid_type=Float)

    def setup(self):
        super().setup()
        wallclock_seconds = 60 * 60 * 8 # 8 hours
        for param in self.inputs.parameters.get_dict()["input_keywords"]:
            if param.startswith("PAL"):
                nprocs = int(param[-1])
        
        metadata = {
            "options": {
                "resources": self.inputs.resources.get_dict(),
                "max_wallclock_seconds": wallclock_seconds,
                "withmpi": False,
                "max_memory_kb": int(3.8 * 1e6 * nprocs),
                "account": "account_xxxx"
            }
        }
        self.ctx.inputs = {'structure': self.inputs.structure, 'code': self.inputs.code, 'parameters': 
        self.inputs.parameters, "metadata": metadata}

    def extract_homo_lumo(self):
        outputs = self.exposed_outputs(OrcaCalculation)
        outputdata = outputs["output_parameters"]
        E_homo = get_homo(outputdata)
        E_lumo = get_lumo(outputdata)
        self.out('homo', E_homo)
        self.out('lumo', E_lumo)

```

I just ran this workchain and was met with this (contracted) error:

```auto
  File "/home/coopy/envs/aiida/lib/python3.10/site-packages/plumpy/workchains.py", line 246, in step
    return True, self._fn(self._workchain)
  File "/home/coopy/onedrive/Research/Batteries/aiida/WorkChains/HomoLumoWorkChain.py", line 104, in extract_homo_lumo
    # outputs = self.get_outputs(OrcaCalculation)
  File "/home/coopy/envs/aiida/lib/python3.10/site-packages/aiida/engine/processes/workchains/restart.py", line 316, in get_outputs
    return self.exposed_outputs(node, self.process_class)
  File "/home/coopy/envs/aiida/lib/python3.10/site-packages/aiida/engine/processes/process.py", line 965, in exposed_outputs
    process_outputs_dict = node.base.links.get_outgoing(link_type=link_types).nested()
AttributeError: type object 'OrcaCalculation' has no attribute 'base'

```

I’ve also tried taking the outputs in the **extract\_homo\_lumo()** method using:

```auto
outputs = self.get_outputs(OrcaCalculation)

```

which also failed with a similar error message.

How should I be extracting the outputs and processing them in other steps? I’ve searched pretty thoroughly through the docs and think this may be a hole in the current documentation. Let me know if I missed something.

Thanks!

---

<div class="post-metadata">

**Author:** ![sphuber](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/sphuber/32/6_2.png) [@sphuber](https://aiida.discourse.group/u/sphuber)\
**Post date:** [May 20, 2024, 10:50am UTC](https://aiida.discourse.group/t/processing-calcjob-outputs-in-baserestartworkchain/391/2 "2024-05-20T10:50:36Z")

</div>

There seems to be something funky with your workchain. The exception is referencing a commented out line:

> [@cote3804](#):
>
> ```auto
> File "/home/coopy/onedrive/Research/Batteries/aiida/WorkChains/HomoLumoWorkChain.py", line 104, in extract_homo_lumo
> # outputs = self.get_outputs(OrcaCalculation)
> 
> ```

That should never happen so I think there is a weird state of your daemon. Did you remember to restart your daemon after editing the workchain file?

Another weird thing is the exception itself claiming that the node doesn’t have the `base` attribute. What version of AiiDA is installed? Do you have multiple virtual environments perhaps and are you running in the correct one?

Either way, there a few ways you can retrieve outputs of a completed process node.

1. Directly: through `node.ouputs.some_link_label`  
2: If you have called `expose_outputs` in the `define` method, you can use `self.exposed_outputs(ProcessClass, namespace=namespace)`.

You are doing this in the version of the workchain you pasted:

```python
outputs = self.exposed_outputs(OrcaCalculation)

```

and that should be correct.

So I think there is nothing wrong with that code, just that your daemon is probably not using the correct version of the file.

---

<div class="post-metadata">

**Author:** ![cote3804](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/cote3804/32/99_2.png) [@cote3804](https://aiida.discourse.group/u/cote3804)\
**Post date:** [May 20, 2024, 3:52pm UTC](https://aiida.discourse.group/t/processing-calcjob-outputs-in-baserestartworkchain/391/3 "2024-05-20T15:52:00Z")

</div>

Hi @sphuber

I generally restart my daemon after changing code. I may not have remembered in this instance, although I have reproduced this exact issue many times after restarting the daemon. That base attribute issue has been persistent.

I only have one venv and it has aiida-core 2.5.1.

I just restarted the daemon with the --reset flag and reran the workflow with this code in the problematic section

```auto
def extract_homo_lumo(self):
        outputs = self.exposed_outputs(OrcaCalculation)

```

and got a new error message:

```auto
  File "/home/coopy/envs/aiida/lib/python3.10/site-packages/plumpy/process_states.py", line 228, in execute
    result = self.run_fn(*self.args, **self.kwargs)
  File "/home/coopy/envs/aiida/lib/python3.10/site-packages/aiida/engine/processes/workchains/workchain.py", line 313, in _do_step
    finished, stepper_result = self._stepper.step()
  File "/home/coopy/envs/aiida/lib/python3.10/site-packages/plumpy/workchains.py", line 295, in step
    finished, result = self._child_stepper.step()
  File "/home/coopy/envs/aiida/lib/python3.10/site-packages/plumpy/workchains.py", line 246, in step
    return True, self._fn(self._workchain)
  File "/home/coopy/onedrive/Research/Batteries/aiida/WorkChains/HomoLumoWorkChain.py", line 103, in extract_homo_lumo
    outputs = self.exposed_outputs(OrcaCalculation)
TypeError: Process.exposed_outputs() missing 1 required positional argument: 'process_class'

```

---

<div class="post-metadata">

**Author:** ![cote3804](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/cote3804/32/99_2.png) [@cote3804](https://aiida.discourse.group/u/cote3804)\
**Post date:** [May 20, 2024, 3:59pm UTC](https://aiida.discourse.group/t/processing-calcjob-outputs-in-baserestartworkchain/391/4 "2024-05-20T15:59:28Z")

</div>

And using the `self.get_outputs(OrcaCalculation)` version after a daemon restart with the --reset flag gives:

```auto
  File "/home/coopy/envs/aiida/lib/python3.10/site-packages/plumpy/process_states.py", line 228, in execute
    result = self.run_fn(*self.args, **self.kwargs)
  File "/home/coopy/envs/aiida/lib/python3.10/site-packages/aiida/engine/processes/workchains/workchain.py", line 313, in _do_step
    finished, stepper_result = self._stepper.step()
  File "/home/coopy/envs/aiida/lib/python3.10/site-packages/plumpy/workchains.py", line 295, in step
    finished, result = self._child_stepper.step()
  File "/home/coopy/envs/aiida/lib/python3.10/site-packages/plumpy/workchains.py", line 246, in step
    return True, self._fn(self._workchain)
  File "/home/coopy/onedrive/Research/Batteries/aiida/WorkChains/HomoLumoWorkChain.py", line 102, in extract_homo_lumo
    outputs = self.get_outputs(OrcaCalculation)
  File "/home/coopy/envs/aiida/lib/python3.10/site-packages/aiida/engine/processes/workchains/restart.py", line 316, in get_outputs
    return self.exposed_outputs(node, self.process_class)
  File "/home/coopy/envs/aiida/lib/python3.10/site-packages/aiida/engine/processes/process.py", line 965, in exposed_outputs
    process_outputs_dict = node.base.links.get_outgoing(link_type=link_types).nested()
AttributeError: type object 'OrcaCalculation' has no attribute 'base'

```

Is this possibly an issue with the aiida-orca plugin?

---

<div class="post-metadata">

**Author:** ![t-reents](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/t-reents/32/82_2.png) [@t-reents](https://aiida.discourse.group/u/t-reents)\
**Post date:** [May 20, 2024, 4:55pm UTC](https://aiida.discourse.group/t/processing-calcjob-outputs-in-baserestartworkchain/391/5 "2024-05-20T16:55:26Z")

</div>

Hi @cote3804!

This is not a bug of the `aiida-orca` plugin. You only pass the `OrcaCalculation` class to the `get_outputs` or `self.exposed_outputs` methods. To extract the outputs, you would need to pass a node of an actual calculation. See the corresponding implementation of the `BaseRestartWorkChain` for reference ([aiida-core/src/aiida/engine/processes/workchains/restart.py at main · aiidateam/aiida-core · GitHub](https://github.com/aiidateam/aiida-core/blob/main/src/aiida/engine/processes/workchains/restart.py#L310-L345)):

```auto
    def get_outputs(self, node) -> Mapping[str, orm.Node]:
        # Comments and doc string removed
        return self.exposed_outputs(node, self.process_class)

    def results(self) -> Optional['ExitCode']:
        # Comments and doc string removed
        node = self.ctx.children[self.ctx.iteration - 1]
        max_iterations = self.inputs.max_iterations.value
        if not self.ctx.is_finished and self.ctx.iteration >= max_iterations:
            self.report(
                f'reached the maximum number of iterations {max_iterations}: '
                f'last ran {self.ctx.process_name}<{node.pk}>'
            )
            return self.exit_codes.ERROR_MAXIMUM_ITERATIONS_EXCEEDED

        self.report(f'work chain completed after {self.ctx.iteration} iterations')
        self._attach_outputs(node)
        return None

    def _attach_outputs(self, node) -> Mapping[str, orm.Node]:
        # Comments and doc string removed
        outputs = self.get_outputs(node)
        existing_outputs = self.node.base.links.get_outgoing(link_type=LinkType.RETURN).all_link_labels()

```

There you can see that the `CalcJob` of the last iteration `node = self.ctx.children[self.ctx.iteration - 1]` is passed to those methods (instead, you are basically passing self.\_process\_class at the moment). Note that `get_output` takes only the node as an input, whereas `exposed_outputs` requires the actual node and the `process_class` as an input.

This being said, changing your code in the following way should fix the issue:

```auto
def extract_homo_lumo(self):
        last_calc = self.ctx.children[self.ctx.iteration - 1]
        outputs = self.exposed_outputs(last_calc, OrcaCalculation)
        outputdata = outputs["output_parameters"]
        E_homo = get_homo(outputdata)
        E_lumo = get_lumo(outputdata)
        self.out('homo', E_homo)
        self.out('lumo', E_lumo)

```

Hope that helps.

---

<div class="post-metadata">

**Author:** ![sphuber](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/sphuber/32/6_2.png) [@sphuber](https://aiida.discourse.group/u/sphuber)\
**Post date:** [May 20, 2024, 6:02pm UTC](https://aiida.discourse.group/t/processing-calcjob-outputs-in-baserestartworkchain/391/6 "2024-05-20T18:02:06Z")

</div>

My bad, I completely read over the fact that the actual node was missing from the `exposed_outputs` call. @t-reents is right and his suggestion should indeed solve it

---

<div class="post-metadata">

**Author:** ![cote3804](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/cote3804/32/99_2.png) [@cote3804](https://aiida.discourse.group/u/cote3804)\
**Post date:** [May 24, 2024, 3:57pm UTC](https://aiida.discourse.group/t/processing-calcjob-outputs-in-baserestartworkchain/391/7 "2024-05-24T15:57:00Z")

</div>

Sorry for the late response. My cluster went down and I didn’t have the chance to test this solution.

This did completely resolve my issues. I think it may still be worth adding an example like this, where the outputs from the process class are processed in the work chain, to the “How to write error resistant workflows” section of the docs.

Thanks for the help!

---

<div class="post-metadata">

**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/aiida/original/1X/c417197d3750949351575d03ed03f5e2ad32e649.png) [@system](https://aiida.discourse.group/u/system)\
**Post date:** [May 30, 2024, 11:57am UTC](https://aiida.discourse.group/t/processing-calcjob-outputs-in-baserestartworkchain/391/8 "2024-05-30T11:57:07Z")

</div>

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.