How to use scp/rsync instead of sftp for transport

Zhishuo · November 18, 2025, 9:34pm

Dear all,

I am setting up the computer within AiiDA on my personal laptop for a remote HPC (bridges2 from PSC), with the configuration as following:

allow_agent: true
compress: true
gss_auth: false
gss_deleg_creds: false
gss_host: bridges2.psc.edu
gss_kex: false
key_filename: /Users/PATH/.ssh/id_rsa_github
key_policy: AutoAddPolicy
load_system_host_keys: false
look_for_keys: true
port: 22
proxy_command: ''
proxy_jump: ''
safe_interval: 30.0
timeout: 60
use_login_shell: true
username: MyUserName

The configuration is about the login node of PSC HPC.
When I did the test, I got the following error:

* Opening connection... [FAILED]: Error while trying to connect to the computer
  Full traceback:
  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/plugins/ssh.py", line 511, in open
      self._sftp = self._client.open_sftp()
                   ^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/client.py", line 608, in open_sftp
      return self._transport.open_sftp_client()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/transport.py", line 1216, in open_sftp_client
      return SFTPClient.from_transport(self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/sftp_client.py", line 169, in from_transport
      chan.invoke_subsystem("sftp")
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 70, in _check
      return func(self, *args, **kwds)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 281, in invoke_subsystem
      self._wait_for_event()
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 1224, in _wait_for_event
      raise e
  paramiko.ssh_exception.SSHException: Channel closed.

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 553, in computer_test
      with transport:
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/transport.py", line 178, in __enter__
      self.open()
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/plugins/ssh.py", line 514, in open
      raise InvalidOperation(
  aiida.common.exceptions.InvalidOperation: Error in ssh transport plugin. This may be due to the remote computer not supporting SFTP. Try setting it up with the aiida.transports:ssh_only transport from the aiida-sshonly plugin instead.
Warning: 1 out of 1 tests failed

It is saying that SFTP is not supported on the login node, while based on the tutorial of the PSC HPC, the files can be transferred via SFTP on the data node (data.bridges2.psc.edu).
So I regenerate the setup of computer regarding that data node, with configuration:

allow_agent: true
compress: true
gss_auth: false
gss_deleg_creds: false
gss_host: data.bridges2.psc.edu
gss_kex: false
key_filename: /Users/PATH/.ssh/id_rsa_github
key_policy: AutoAddPolicy
load_system_host_keys: false
look_for_keys: true
port: 22
proxy_command: ''
proxy_jump: ''
safe_interval: 30.0
timeout: 60
use_login_shell: true
username: MyUserName

And I got the following error:

* Opening connection... [OK]
* Checking for spurious output... /Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/plugins/ssh.py:674: AiidaDeprecationWarning: `chdir()` is deprecated and will be removed in the next major version. Use absolute paths instead. (this will be removed in v3)
  warn_deprecation(
[Failed]: TimeoutError:
  Full traceback:
  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 697, in recv
      out = self.in_buffer.read(nbytes, self.timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/buffered_pipe.py", line 154, in read
      raise PipeTimeout()
  paramiko.buffered_pipe.PipeTimeout

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 564, in computer_test
      success, message = test(
                         ^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 78, in _computer_test_no_unexpected_output
      retval, stdout, stderr = transport.exec_command_wait('echo -n')
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/transport.py", line 500, in exec_command_wait
      retval, stdout_bytes, stderr_bytes = self.exec_command_wait_bytes(
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/plugins/ssh.py", line 1552, in exec_command_wait_bytes
      stdout_bytes.append(stdout.read())
                          ^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/file.py", line 184, in read
      new_data = self._read(self._DEFAULT_BUFSIZE)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 1359, in _read
      return self.channel.recv(size)
             ^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 699, in recv
      raise socket.timeout()
  TimeoutError
* Getting number of jobs from scheduler... [Failed]: TimeoutError:
  Full traceback:
  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 697, in recv
      out = self.in_buffer.read(nbytes, self.timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/buffered_pipe.py", line 154, in read
      raise PipeTimeout()
  paramiko.buffered_pipe.PipeTimeout

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 564, in computer_test
      success, message = test(
                         ^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 58, in _computer_test_get_jobs
      found_jobs = scheduler.get_jobs(as_dict=True)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/schedulers/plugins/bash.py", line 52, in get_jobs
      retval, stdout, stderr = self.transport.exec_command_wait(self._get_joblist_command(jobs=jobs, user=user))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/transport.py", line 500, in exec_command_wait
      retval, stdout_bytes, stderr_bytes = self.exec_command_wait_bytes(
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/plugins/ssh.py", line 1552, in exec_command_wait_bytes
      stdout_bytes.append(stdout.read())
                          ^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/file.py", line 184, in read
      new_data = self._read(self._DEFAULT_BUFSIZE)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 1359, in _read
      return self.channel.recv(size)
             ^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 699, in recv
      raise socket.timeout()
  TimeoutError
* Determining remote user name... [Failed]: TimeoutError:
  Full traceback:
  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 697, in recv
      out = self.in_buffer.read(nbytes, self.timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/buffered_pipe.py", line 154, in read
      raise PipeTimeout()
  paramiko.buffered_pipe.PipeTimeout

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 564, in computer_test
      success, message = test(
                         ^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 115, in _computer_get_remote_username
      remote_user = transport.whoami()
                    ^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/transport.py", line 858, in whoami
      retval, username, stderr = self.exec_command_wait(command)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/transport.py", line 500, in exec_command_wait
      retval, stdout_bytes, stderr_bytes = self.exec_command_wait_bytes(
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/plugins/ssh.py", line 1552, in exec_command_wait_bytes
      stdout_bytes.append(stdout.read())
                          ^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/file.py", line 184, in read
      new_data = self._read(self._DEFAULT_BUFSIZE)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 1359, in _read
      return self.channel.recv(size)
             ^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 699, in recv
      raise socket.timeout()
  TimeoutError
* Creating and deleting temporary file... [Failed]: TimeoutError:
  Full traceback:
  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 697, in recv
      out = self.in_buffer.read(nbytes, self.timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/buffered_pipe.py", line 154, in read
      raise PipeTimeout()
  paramiko.buffered_pipe.PipeTimeout

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 564, in computer_test
      success, message = test(
                         ^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 135, in _computer_create_temp_file
      workdir = authinfo.get_workdir().format(username=transport.whoami())
                                                       ^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/transport.py", line 858, in whoami
      retval, username, stderr = self.exec_command_wait(command)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/transport.py", line 500, in exec_command_wait
      retval, stdout_bytes, stderr_bytes = self.exec_command_wait_bytes(
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/plugins/ssh.py", line 1552, in exec_command_wait_bytes
      stdout_bytes.append(stdout.read())
                          ^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/file.py", line 184, in read
      new_data = self._read(self._DEFAULT_BUFSIZE)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 1359, in _read
      return self.channel.recv(size)
             ^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 699, in recv
      raise socket.timeout()
  TimeoutError
* Checking for possible delay from using login shell... [Failed]: TimeoutError:
  Full traceback:
  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 697, in recv
      out = self.in_buffer.read(nbytes, self.timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/buffered_pipe.py", line 154, in read
      raise PipeTimeout()
  paramiko.buffered_pipe.PipeTimeout

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 564, in computer_test
      success, message = test(
                         ^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 221, in _computer_use_login_shell_performance
      timing_false = time_use_login_shell(authinfo, auth_params_clone, False, iterations)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 192, in time_use_login_shell
      transport.whoami()
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/transport.py", line 858, in whoami
      retval, username, stderr = self.exec_command_wait(command)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/transport.py", line 500, in exec_command_wait
      retval, stdout_bytes, stderr_bytes = self.exec_command_wait_bytes(
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/plugins/ssh.py", line 1552, in exec_command_wait_bytes
      stdout_bytes.append(stdout.read())
                          ^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/file.py", line 184, in read
      new_data = self._read(self._DEFAULT_BUFSIZE)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 1359, in _read
      return self.channel.recv(size)
             ^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/paramiko/channel.py", line 699, in recv
      raise socket.timeout()
  TimeoutError
Warning: 5 out of 6 tests failed

It seems that the sftp connect was established successfully, but the data node does not allow any operations.

I also checked that, there is one way to use only scp/rsync for file transfer, with the module “aiida_onlyssh”. However, I got a severe error about the version mismatch between the libraries in my Conda environment and the aiida_onlyssh software, as it was developed 5 years ago.

I am wondering if it is possible to disable the sftp file transfer when setting up the computer within AiiDA?
As I have checked, the file transfer with rsync is ok.
I truly appreciate it if anyone could help me out.

I am using AiiDA version 2.7.1, and the no-password ssh connect has been setup between my local laptop and PSC HPC, and the AiiDA was well configured on my local laptop:

 ✔ version:     AiiDA v2.7.1
 ✔ config:      /Users/PATH/.aiida
 ✔ profile:     PSAL_dos
 ✔ storage:     Storage for ...
Warning: RabbitMQ v4.1.4 is not supported and will cause unexpected problems!
Warning: It can cause long-running workflows to crash and jobs to be submitted multiple times.
Warning: See https://github.com/aiidateam/aiida-core/wiki/RabbitMQ-version-to-use for details.
 ✔ broker:      RabbitMQ v4.1.4 ... 
 ✔ daemon:      Daemon

Best
Zhishuo

mbercx · November 20, 2025, 11:43pm

Hi @Zhishuo! I’m not sure if this is possible at the moment to be honest. I’m going to ping @ali-khosravi, our transport expert, but he’s on holiday at the moment.

I wasn’t even aware of aiida-sshonly. Seems the developer was running into a similar issue as you, and decided to develop their own transport plugin. But indeed, the work was done some time ago and I see the package hasn’t been updated for aiida-core v2. It might be worthwhile to do so, however.

rikigigi · November 21, 2025, 11:54am

I think the new async plugin with the openssh backend should be able to to the job, after configuring ssh properly for your host in your .ssh/config

Riccardo Bertossa

Zhishuo · November 21, 2025, 1:37pm

Thank you Marnik for your response.

Zhishuo · November 21, 2025, 1:51pm

Hello Riccardo,
Thank you very much for your suggestion.
I tried to setup it up using core.ssh_acrync, for which my config file is:

label: HPCACCESSPSC_verdi_async
description: PSC in ACCESS
hostname: bridges2.psc.edu
transport: core.ssh_async
scheduler: core.slurm
work_dir: "/ocean/projects/PROJECT/USERNAME"
mpirun_command: "srun -n {tot_num_mpiprocs}"
mpiprocs_per_machine: 128

And I tried “verdi computer setup --config HPCACCESSPSC_verdi_async.yaml”, and the interaction is as following:

Report: enter ? for help.
Report: enter ! to ignore the default and set no value.
Shebang line (first line of each script, starting with #!) [#!/bin/bash]:
Default amount of memory per machine (kB).: 265000000
Escape CLI arguments in double quotes [y/N]: y
Success: Computer<19> HPCACCESSPSC_verdi_async created
Report: Note: before the computer can be used, it has to be configured with the command:
Report:   verdi -p PSAL_dos computer configure core.ssh_async HPCACCESSPSC_verdi_async

Then I configured it, “verdi -p PSAL_dos computer configure core.ssh_async HPCACCESSPSC_verdi_async”, with the procedure:

Report: enter ? for help.
Report: enter ! to ignore the default and set no value.
Host as in 'ssh <HOST>' (needs to be a password-less setup in your ssh config) [bridges2.psc.edu]:
Maximum number of concurrent I/O operations [8]:
Local script to run *before* opening connection (path) [None]:
Type of async backend to use, `asyncssh` or `openssh` [asyncssh]: openssh
Use login shell when executing command [Y/n]: Y
Connection cooldown time (s) [30.0]:
Report: Configuring computer HPCACCESSPSC_verdi_async for user mypersonalemail@gmail.com.
Success: HPCACCESSPSC_verdi_async successfully configured for mypersonalemail@gmail.com

The wired thing is that the configuration is for my personal gmail, which is basic the user information of my local laptop, and I can not specify the user name for the HPCs.
I tried to do verdi computer test,

Report: Testing computer<HPCACCESSPSC_verdi_async> for user<mypersonalemail@gmail.com>...
* Opening connection... Warning: RabbitMQ v4.1.4 is not supported and will cause unexpected problems!
Warning: It can cause long-running workflows to crash and jobs to be submitted multiple times.
Warning: See https://github.com/aiidateam/aiida-core/wiki/RabbitMQ-version-to-use for details.
[OK]
Zhishuo@bridges2.psc.edu's password:

It seems that the username is the username on my local laptop.
I tried to re-configure the username “verdi computer configure core.ssh_async -u “HPCusername” HPCACCESSPSC_verdi_async”, and I got:

Usage: verdi computer configure core.ssh_async [OPTIONS] COMPUTER
Try 'verdi computer configure core.ssh_async --help' for help.

Error: Invalid value for '-u' / '--user': User 'HPCusername' not found

Could you please give me a hand on how to provide the login information (username and path of the private ssh-key) of HPCs?

Best
Zhishuo

giovannipizzi · November 21, 2025, 8:40pm

Hi, asyncssh+openssh should use the configuration in your ~/.ssh./config file. So, if you are able to do ssh bridges2.psc.edu without specifying username and password, also AiiDA should be able to do it.

Do you also have a prompt for a password if you do ssh bridges2.psc.edu? (when running this command without specifying a username). I would guess so. Then, configure accordingly your ~/.ssh/config until the password is not required. I.e. add the following lines to it (possibly create the file if it does not exist):

Host bridges2.psc.edu
  Hostname bridges2.psc.edu
  User <YOUR_USERNAME_ON_PSC>
  IdentityFile <PATH_TO_SSH_KEY>

IdentityFile is optional, but ideally you should put the absolute path to your SSH key that allows you to login without a password.

ali-khosravi · November 24, 2025, 7:11am

Hi @Zhishuo , Seems like the email is taken from your profile setup.
I’d follow @giovannipizzi suggestion. In case your HPC strictly requires you to login with manually entering password every time, let us know so that we’ll suggest further solutions on that.

Zhishuo · November 25, 2025, 11:38am

Hello Giovanni,

Thank you very much for your suggestion.
It worked for the connection with ssh to the PSC.
The previous problem is that I set a name different from the hostname in the config file:

Host HPCACCESSPSC
        HostName bridges2.psc.edu
        User USERNAME
        IdentityFile ~/.ssh/id_rsa_github

I changed it into the hostname:

Host bridges2.psc.edu
        HostName bridges2.psc.edu
        User USERNAME
        IdentityFile ~/.ssh/id_rsa_github

However, when I tested the computer with “verdi computer test HPCACCESSPSC_verdi_async”, I got the error as following:

* Opening connection... Warning: RabbitMQ v4.1.4 is not supported and will cause unexpected problems!
Warning: It can cause long-running workflows to crash and jobs to be submitted multiple times.
Warning: See https://github.com/aiidateam/aiida-core/wiki/RabbitMQ-version-to-use for details.
[OK]
* Checking for spurious output... [OK]
* Getting number of jobs from scheduler... [OK]: 1854 jobs found in the queue
* Determining remote user name... [OK]: Username
* Creating and deleting temporary file... [Failed]: OSError: Error while uploading file /var/folders/cz/hcgvrw9502sg1lqf51hzw6pc0000gq/T/tmpmt_dtvs2: {'subsystem request failed on channel 0\r\nscp: Connection closed\r\n'}
  Full traceback:
  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/plugins/ssh_async.py", line 529, in putfile_async
      await self.async_backend.put(
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/plugins/async_backend.py", line 652, in put
      raise OSError({stderr})
  OSError: {'subsystem request failed on channel 0\r\nscp: Connection closed\r\n'}

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 564, in computer_test
      success, message = test(
                         ^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/cmdline/commands/cmd_computer.py", line 145, in _computer_create_temp_file
      transport.putfile(tempf.name, remote_file_path)
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/transport.py", line 1828, in putfile
      return self.run_command_blocking(self.putfile_async, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/transport.py", line 1804, in run_command_blocking
      return loop.run_until_complete(func(*args, **kwargs))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/engine/runners.py", line 160, in run_until_complete
      return self._loop.run_until_complete(future)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/nest_asyncio.py", line 98, in run_until_complete
      return f.result()
             ^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/asyncio/futures.py", line 203, in result
      raise self._exception.with_traceback(self._exception_tb)
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/asyncio/tasks.py", line 277, in __step
      result = coro.send(None)
               ^^^^^^^^^^^^^^^
    File "/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/plugins/ssh_async.py", line 534, in putfile_async
      raise OSError(f'Error while uploading file {localpath}: {exc}')
  OSError: Error while uploading file /var/folders/cz/hcgvrw9502sg1lqf51hzw6pc0000gq/T/tmpmt_dtvs2: {'subsystem request failed on channel 0\r\nscp: Connection closed\r\n'}
* Checking for possible delay from using login shell... [OK]
Warning: 1 out of 6 tests failed

It seems that the “scp” could not work well.
I tried to use “scp” in command line:

(aiida) Zhishuo@Mac Phonon_splits % scp Check_log bridges2.psc.edu:
subsystem request failed on channel 0
scp: Connection closed

But if I add “-O”, it works fine:

(aiida) Zhishuo@Mac Phonon_splits % scp -O Check_log bridges2.psc.edu:
Check_log                                                                                                                                                             100%   68KB 215.6KB/s   00:00

Do you have any idea to solve it?

Best
Zhishuo

ali-khosravi · November 25, 2025, 1:13pm

Hi Zhishuo,

Thanks for trying this out and reporting your solution, I wasn’t aware of flag -O. From scp manual I read:

-O Use the legacy SCP protocol for file transfers instead of the SFTP protocol. Forcing the use of the SCP protocol may be necessary for servers that do not implement SFTP, for backwards-compatibility for particular filename wildcard patterns and for expanding paths with a ‘~’ prefix for older SFTP servers.

Since -O solves the error, then the fastest way is to adopt the source code of aiida installation in your computer. (Unfortunately at the moment we the configuration doesn’t accept custom flags)

Solution #1, fast and dirty

Take the path from in your terminal run, just to be sure:

$ python -c "import aiida; print(aiida.__file__)"
/Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/__init__.py

Then open this file:

$ open /Users/Zhishuo/anaconda3/envs/aiida/lib/python3.11/site-packages/aiida/transports/plugins/async_backend.py

Replace all options = [] with options = ['-O'] , save the file and exit.

$ verdi daemon restart
$ verdi computer test HPCACCESSPSC_verdi_async

Solution #2, clean, but no guarantee that it works:

add an alias to your .bashrc, .zshrc, etc.

alias scp=‘scp -O’

Zhishuo · November 25, 2025, 1:53pm

Hello Ali,

Thank you very much for the instruction.

I tried Solution #1, and it works well with all the 6 tests past.

Actually, my colleague tried Solution #2, which resulted in other issues (Sorry I do not have the details from him).

Best
Zhishuo

ali-khosravi · November 26, 2025, 10:48am

Glad to hear that worked! let us know if you had any other issues.

Cheers!

Topic		Replies	Views
paramiko.ssh_exception.SSHException: No authentication methods available New to AiiDA question	7	866	February 13, 2024
Configure Aiida to use remote HPC General Usage question	40	742	February 14, 2024
paramiko.ssh_exception.NoValidConnectionsError New to AiiDA	9	1052	September 14, 2023
Error while trying to connect to the computer New to AiiDA	2	118	February 28, 2024
Accessing clusters with 2-factor authentication (Sigma2) using AiiDA General Usage	7	161	March 13, 2025

How to use scp/rsync instead of sftp for transport

Related topics