# Strange behaviour of calcfunctions when many links are generated in outputs

**URL:** <https://aiida.discourse.group/t/strange-behaviour-of-calcfunctions-when-many-links-are-generated-in-outputs/579>\
**Category:** General Usage\
**Created:** [March 12, 2025, 10:07am UTC](https://aiida.discourse.group/t/strange-behaviour-of-calcfunctions-when-many-links-are-generated-in-outputs/579 "2025-03-12T10:07:52Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![bastonero](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/bastonero/32/32_2.png) [@bastonero](https://aiida.discourse.group/u/bastonero)\
**Post date:** [March 12, 2025, 10:07am UTC](https://aiida.discourse.group/t/strange-behaviour-of-calcfunctions-when-many-links-are-generated-in-outputs/579/1 "2025-03-12T10:07:52Z")

</div>

Dear all, I am experiencing a weird behaviour of `calcfunction` for a method I wrote to extract structures. This method can generate ~10^3 – 10^4 output nodes, and in the provenance graph something very strange happens:

```auto
...
        └── FirstWorkChain<5065414> Waiting [2:if_(...)]
            ├── SecondWorkChain<5065417> Finished [0] [2:run_...]
            │ ├── ThirdWorkChain<5065418> Finished [0] [4:if_(...)]
            │ │ ...
            │ ├── get_structures<5065474> Running [0]
            │ └── get_structures<5066148> Finished [0]
...

```

In the `SecondWorkChain` where the `get_structures` is called there is no loop that might call a second time the calcfunction. It seems that the first calcfunction job finishes successfully, but somehow does not lock completely (as i guess is still creating the output links). At this point, somehow a second call is made to the same function and it finished correctly. This is very weird, and I also experienced this behaviour multiple times with even more than 2 calls (even 10 at times).

Is there a problem or a known limitation in having so many output nodes? How to circumvent this?

Thanks a lot for any pointer.

---

<div class="post-metadata">

**Author:** ![giovannipizzi](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/giovannipizzi/32/17_2.png) [@giovannipizzi](https://aiida.discourse.group/u/giovannipizzi)\
**Post date:** [March 13, 2025, 8:04pm UTC](https://aiida.discourse.group/t/strange-behaviour-of-calcfunctions-when-many-links-are-generated-in-outputs/579/2 "2025-03-13T20:04:09Z")

</div>

Hi Lorenzo! I don’t think there should be a limitation. Which version of AiiDA-core and main dependencies are you using? Which backend? (PSQL+dostore?) Would you be able to make a reproducible mock example so we can debug more easily? Pinging @agoscinski @geiger_j @jusong.yu

---

<div class="post-metadata">

**Author:** ![bastonero](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/bastonero/32/32_2.png) [@bastonero](https://aiida.discourse.group/u/bastonero)\
**Post date:** [March 13, 2025, 8:30pm UTC](https://aiida.discourse.group/t/strange-behaviour-of-calcfunctions-when-many-links-are-generated-in-outputs/579/3 "2025-03-13T20:30:34Z")

</div>

Thanks Giovanni for the help! It is PostgreSQL + RabbitMQ. The aiida version i am using is 2.4.0 with a “special patch” from Sebastiaan from a while ago (this is the discussion: [Excepted workchains, due to strange error from kiwipy/plumpy?](https://aiida.discourse.group/t/excepted-workchains-due-to-strange-error-from-kiwipy-plumpy/198)).  
I don’t know whether that fix has now been implemented in the latest `main` branch of aiida-core.  
The error though seems kind of random, depending also on the rest of the daemon workload, and even running using caching to skip the steps the behaviour was quite different each time. So, I guess it’s probably related to the version of aiida-core i am using, and by the fact i am running quite heavily on my workstation (?).

---

<div class="post-metadata">

**Author:** ![giovannipizzi](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/giovannipizzi/32/17_2.png) [@giovannipizzi](https://aiida.discourse.group/u/giovannipizzi)\
**Post date:** [March 13, 2025, 8:48pm UTC](https://aiida.discourse.group/t/strange-behaviour-of-calcfunctions-when-many-links-are-generated-in-outputs/579/4 "2025-03-13T20:48:23Z")

</div>

Mmm, it might be. A few things have been fixed recently in the current main branch. If you could try to see if you can reproduce in a test environment with master (soon to be released as 2.7) that would be great.

---

<div class="post-metadata">

**Author:** ![sphuber](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/sphuber/32/6_2.png) [@sphuber](https://aiida.discourse.group/u/sphuber)\
**Post date:** [March 14, 2025, 10:38am UTC](https://aiida.discourse.group/t/strange-behaviour-of-calcfunctions-when-many-links-are-generated-in-outputs/579/5 "2025-03-14T10:38:50Z")

</div>

This is very likely due to the bugs in the engine that could result in multiple daemon workers running the same task (exactly as in the thread you linked). This has been fixed and should be released with v2.7 if I remember correctly.

---

<div class="post-metadata">

**Author:** ![bastonero](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/bastonero/32/32_2.png) [@bastonero](https://aiida.discourse.group/u/bastonero)\
**Post date:** [March 14, 2025, 11:26am UTC](https://aiida.discourse.group/t/strange-behaviour-of-calcfunctions-when-many-links-are-generated-in-outputs/579/6 "2025-03-14T11:26:39Z")

</div>

Thanks a lot @sphuber !

---

<div class="post-metadata">

**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/aiida/original/1X/c417197d3750949351575d03ed03f5e2ad32e649.png) [@system](https://aiida.discourse.group/u/system)\
**Post date:** [March 20, 2025, 7:26am UTC](https://aiida.discourse.group/t/strange-behaviour-of-calcfunctions-when-many-links-are-generated-in-outputs/579/7 "2025-03-20T07:26:48Z")

</div>

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.
