Is there a Desktop Job Scheduler for MacOS

Hello everyone,

I have written a workchain that uses software on my local computer(Apple M1, macOS Ventura 13.5.2). I intend to use this to run workchains on multiple files in parallel, which would lead to many running instances of the local software, so I would like to use a scheduler that isn’t direct.

I have been having trouble finding a scheduler available to submit these calculations to. Does anyone have any advice on schedulers for Mac (with available plugins ideally)


The direct scheduler does not work?

I think the issue is that once you start running a bunch of work chains in parallel on your local system, the CPU’s get overwhelmed quickly? As far as I know you can’t limit the number of active processes on a computer.

It seems you can install slurm with brew, but that might be overkill and a lot of work. It might also be an interesting use case of the HyperQueue scheduler, for which there is a plugin, but I’d have to try that out myself to see how (or if) it works.

If you can run one or a couple work chains together without issue, you could also consider writing a submission controller, so you could e.g. always keep two active work chains. Let me know if you need help with that, the documentation is still lacking.

Seems like the submission controller will be a good option for me to explore. I’ll see about getting that implemented, thanks!

1 Like

Great, let me know if you need any help! A few notes:

  1. I’ve made some more recent changes locally (see here). I’ll try to integrate these quickly and do a release if you think they’re useful.
  2. I haven’t updated the QE example yet since moving to pydantic, apologies. Below is a rough outline of what a submission controller with the changes in [1] should look like (pseudo-code):
from aiida import orm
from aiida_submission_controller import FromGroupSubmissionController

from your_module import YourWorkChain

class YourWorkChainController(FromGroupSubmissionController):
    """A SubmissionController for submitting `YourWorkChain`s."""
    code: str
    other_parameter: int

    def get_inputs_and_processclass_from_extras(self, extras_values, dry_run=False):
        """Return inputs and process class for the submission of this specific process."""
        parent_node = self.get_parent_node_from_extras(extras_values)

        builder = YourWorkChain.get_builder()

        # Provide the inputs for the builder based on the `parent_node` and controller settings.

        return builder

You can then submit a batch of work chains using:

your_controller = YourWorkChainController(
    group_label='your/workchain/group',  # label of the group you want to put your work chains in
    parent_group_label='parent/group/label',  # label of the parent group with e.g. `StructureData`