# Is there a Desktop Job Scheduler for MacOS

**URL:** <https://aiida.discourse.group/t/is-there-a-desktop-job-scheduler-for-macos/95>\
**Category:** New to AiiDA\
**Tags:** question\
**Created:** [September 11, 2023, 1:52pm UTC](https://aiida.discourse.group/t/is-there-a-desktop-job-scheduler-for-macos/95 "2023-09-11T13:52:49Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![kmlefran](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/kmlefran/32/48_2.png) [@kmlefran](https://aiida.discourse.group/u/kmlefran)\
**Post date:** [September 11, 2023, 1:52pm UTC](https://aiida.discourse.group/t/is-there-a-desktop-job-scheduler-for-macos/95/1 "2023-09-11T13:52:49Z")

</div>

Hello everyone,

I have written a workchain that uses software on my local computer(Apple M1, macOS Ventura 13.5.2). I intend to use this to run workchains on multiple files in parallel, which would lead to many running instances of the local software, so I would like to use a scheduler that isn’t direct.

I have been having trouble finding a scheduler available to submit these calculations to. Does anyone have any advice on schedulers for Mac (with available plugins ideally)

Thanks,

---

<div class="post-metadata">

**Author:** ![ltalirz](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/ltalirz/32/11_2.png) [@ltalirz](https://aiida.discourse.group/u/ltalirz)\
**Post date:** [September 11, 2023, 2:14pm UTC](https://aiida.discourse.group/t/is-there-a-desktop-job-scheduler-for-macos/95/2 "2023-09-11T14:14:27Z")

</div>

The `direct` scheduler does not work?

---

<div class="post-metadata">

**Author:** ![mbercx](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/mbercx/32/345_2.png) [@mbercx](https://aiida.discourse.group/u/mbercx)\
**Post date:** [September 11, 2023, 2:22pm UTC](https://aiida.discourse.group/t/is-there-a-desktop-job-scheduler-for-macos/95/3 "2023-09-11T14:22:27Z")

</div>

I think the issue is that once you start running a bunch of work chains in parallel on your local system, the CPU’s get overwhelmed quickly? As far as I know you can’t limit the number of active processes on a computer.

It seems you can [install slurm with brew](https://formulae.brew.sh/formula/slurm), but that might be overkill and a lot of work. It might also be an interesting use case of the [HyperQueue scheduler](https://github.com/It4innovations/hyperqueue), for which there is a [plugin](https://github.com/aiidateam/aiida-hyperqueue), but I’d have to try that out myself to see how (or if) it works.

If you can run one or a couple work chains together without issue, you could also consider writing a [submission controller](https://github.com/aiidateam/aiida-submission-controller), so you could e.g. always keep two active work chains. Let me know if you need help with that, the documentation is still lacking.

---

<div class="post-metadata">

**Author:** ![kmlefran](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/kmlefran/32/48_2.png) [@kmlefran](https://aiida.discourse.group/u/kmlefran)\
**Post date:** [September 11, 2023, 2:39pm UTC](https://aiida.discourse.group/t/is-there-a-desktop-job-scheduler-for-macos/95/4 "2023-09-11T14:39:17Z")

</div>

> [@mbercx](#):
>
> plugin

Seems like the submission controller will be a good option for me to explore. I’ll see about getting that implemented, thanks!

---

<div class="post-metadata">

**Author:** ![mbercx](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/mbercx/32/345_2.png) [@mbercx](https://aiida.discourse.group/u/mbercx)\
**Post date:** [September 11, 2023, 3:20pm UTC](https://aiida.discourse.group/t/is-there-a-desktop-job-scheduler-for-macos/95/5 "2023-09-11T15:20:04Z")

</div>

Great, let me know if you need any help! A few notes:

1. I’ve made some more recent changes locally (see [here](https://github.com/aiidateam/aiida-submission-controller/pull/16)). I’ll try to integrate these quickly and do a release if you think they’re useful.
2. I haven’t updated the QE example yet since moving to `pydantic`, apologies. Below is a rough outline of what a submission controller with the changes in [1] should look like (pseudo-code):

```auto
from aiida import orm
from aiida_submission_controller import FromGroupSubmissionController

from your_module import YourWorkChain

class YourWorkChainController(FromGroupSubmissionController):
    """A SubmissionController for submitting `YourWorkChain`s."""
    code: str
    other_parameter: int

    def get_inputs_and_processclass_from_extras(self, extras_values, dry_run=False):
        """Return inputs and process class for the submission of this specific process."""
        parent_node = self.get_parent_node_from_extras(extras_values)

        builder = YourWorkChain.get_builder()

        # Provide the inputs for the builder based on the `parent_node` and controller settings.

        return builder

```

You can then submit a batch of work chains using:

```auto
your_controller = YourWorkChainController(
    unique_extra_keys=('unique_id'),
    code=orm.load_code('your_code@localhost'),
    group_label='your/workchain/group', # label of the group you want to put your work chains in
    max_concurrent=max_concurrent, 
    parent_group_label='parent/group/label', # label of the parent group with e.g. `StructureData`
)
supercon_controller.submit_new_batch(verbose=True)

```

---

<div class="post-metadata">

**Author:** ![jusong.yu](https://yyz2.discourse-cdn.com/free1/user_avatar/aiida.discourse.group/jusong.yu/32/3_2.png) [@jusong.yu](https://aiida.discourse.group/u/jusong.yu)\
**Post date:** [November 27, 2024, 10:51am UTC](https://aiida.discourse.group/t/is-there-a-desktop-job-scheduler-for-macos/95/6 "2024-11-27T10:51:49Z")

</div>

For the lightweight scheduler that easy to run and setup for local machine, as mention, hyperqueue can be a nice solution. Check [Implementing the Flux scheduler from LLNL - #4 by jusong.yu](https://aiida.discourse.group/t/implementing-the-flux-scheduler-from-llnl/473/4) for details.