Expose outputs from ShellJob

General Usage
1

Hello everyone,

I have a question regarding the aiida-shell plugin. I created a script to run the ‘lobster’ code with aiida-shell within my workchain. At the end of the process, I extracted all the desired files and passed the resulting node to expose its outputs to the main parent node. Here is the relevant part of my code:

…
outputs=[‘lobsterout’, ‘POSCAR.lobster’, ‘DOSCAR.lobster’, ‘COHPCAR.lobster’, ‘COOPCAR.lobster’, ‘COBICAR.lobster’, ‘ICOHPLIST.lobster’, ‘ICOOPLIST.lobster’, ‘ICOBILIST.lobster’,
‘CHARGE.lobster’],
…
self.out_many(self.exposed_outputs(lobster_node, ShellJob, namespace=‘lobster_results’))

And in the beginning of the class:

spec.expose_outputs(ShellJob, namespace=‘lobster_results’)

The issue I am facing is that it is only exposing the ‘remote_folder’ and the ‘retrieved’ outputs, while ignoring all the other SingleFileData outputs that are present in the ‘lobster_node’.

Can anyone help me understand why this is happening and how I can ensure all the outputs are correctly exposed?

Thank you!

2

Hi @DoriniTT , exposing outputs only works for outputs that are explicitly declared in the process spec. However, the ShellJob does not do this but just marks the output namespace as dynamic:

This just means that any output can be registered. The reason for that is because the ShellJob cannot know in advance which outputs will be generated because it depends on the inputs.

That being said, you don’t need to rely on exposing explicit outputs. You can simply make the output namespace of your workchain dynamic as well, and then add the outputs of the ShellJob with code something like the following:

def add_outputs(self)
    shell_job_node = self.ctx.shell_job
    for output in shell_job_node.base.links.get_outgoing().all():
        self.out(output.link_label, output.node)

I think that should do it.

3

Hi Sebastiaan,

Thank you for the explanation!

Based on what you’ve shared, should I also include spec.outputs(output.link_label) in the define function of my class? When I tried to run it in my workchain, I encountered the following error:

ValueError: Error validating output 'uuid: 77f026f5-0b39-407e-b13b-e98c0a9d0429 (pk: 426)' for port 'outputs': Unexpected ports {'remote_folder': <RemoteData: uuid: 77f026f5-0b39-407e-b13b-e98c0a9d0429 (pk: 426)>}, for a non dynamic namespace

I’m not sure what is causing this issue. Could you please help me understand what’s going wrong?

The list that I am iterating in shell_job_node.base.links.get_outgoing().all() is as follows:

[
    LinkTriple(node=<RemoteData: uuid: 77f026f5-0b39-407e-b13b-e98c0a9d0429 (pk: 426)>, link_type=<LinkType.CREATE: 'create'>, link_label='remote_folder'),
    LinkTriple(node=<FolderData: uuid: b092be4c-f328-4bc6-b862-146cb7bfad86 (pk: 427)>, link_type=<LinkType.CREATE: 'create'>, link_label='retrieved'),
    LinkTriple(node=<SinglefileData: uuid: 554eb4dc-bf1c-41e5-a939-b4e217583961 (pk: 428)>, link_type=<LinkType.CREATE: 'create'>, link_label='lobsterout'),
    LinkTriple(node=<SinglefileData: uuid: 352cfec5-8264-4af4-98b6-5077e6f55ab4 (pk: 429)>, link_type=<LinkType.CREATE: 'create'>, link_label='POSCAR_lobster'),
    LinkTriple(node=<SinglefileData: uuid: 1a369dd4-c075-4233-9ee2-929333678ada (pk: 430)>, link_type=<LinkType.CREATE: 'create'>, link_label='DOSCAR_lobster'),
    LinkTriple(node=<SinglefileData: uuid: 530f5fb5-b071-4cab-8561-78293bc789b9 (pk: 431)>, link_type=<LinkType.CREATE: 'create'>, link_label='COHPCAR_lobster'),
    LinkTriple(node=<SinglefileData: uuid: fb6df07d-61a0-4dd2-bdad-52e6c5d56300 (pk: 432)>, link_type=<LinkType.CREATE: 'create'>, link_label='COOPCAR_lobster'),
    LinkTriple(node=<SinglefileData: uuid: a0b48276-1ec0-4d27-b9d5-054374dd653c (pk: 433)>, link_type=<LinkType.CREATE: 'create'>, link_label='COBICAR_lobster'),
    LinkTriple(node=<SinglefileData: uuid: 09700130-a107-4689-be6d-9b25e8dd8d3b (pk: 434)>, link_type=<LinkType.CREATE: 'create'>, link_label='ICOHPLIST_lobster'),
    LinkTriple(node=<SinglefileData: uuid: f61e281d-dddc-44aa-973d-487beae3edf9 (pk: 435)>, link_type=<LinkType.CREATE: 'create'>, link_label='ICOOPLIST_lobster'),
    LinkTriple(node=<SinglefileData: uuid: 88c9d025-e473-4472-a52b-9ea651d56579 (pk: 436)>, link_type=<LinkType.CREATE: 'create'>, link_label='ICOBILIST_lobster'),
    LinkTriple(node=<SinglefileData: uuid: 1e473ac1-3917-40b7-ba05-002c35de05be (pk: 437)>, link_type=<LinkType.CREATE: 'create'>, link_label='CHARGE_lobster'),
    LinkTriple(node=<SinglefileData: uuid: f1e2836b-9ecf-4896-9733-ba2ab900080a (pk: 438)>, link_type=<LinkType.CREATE: 'create'>, link_label='stderr'),
    LinkTriple(node=<SinglefileData: uuid: eefe99ba-819b-47cf-8489-450ca4d473ce (pk: 439)>, link_type=<LinkType.CREATE: 'create'>, link_label='stdout')
]

Thanks a lot!

4

As Sebastiaan mentioned, you need to make the output namespace of your workchain dynamic.

Add this in your define method

spec.outputs.dynamic = True
5

I see, it worked!

Thanks a lot!

6

Glad it worked.

This won’t work, because define is a classmethod and so is executed once for the class itself. Whereas the outputs are only defined for an actual run of the class (i.e. an instance). The define method specified what should happen in general, but since it cannot know what outputs would be generated in general (since they are dynamic and dependent on the inputs) you cannot define them explicitly. This is no problem though as you can just mark the output as dynamic as Xing showed. This tells AiiDA that you don’t know exactly which outputs will be generated, but any output will be accepted.

1 Like
7

I see! Thanks for the explanation!

8

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.