Xing
October 30, 2023, 9:06am
1
The CSCS eiger now uses a new scratch filesystem. To use this new filesystem, I can create a new aiida computer. The downsides of this are:
Set up the computer and related codes again.
Can not use the remote_folder because Remote copy between two different machines is not implemented yet. This is a critical issue in my case.
My question is: is there a method to simply update the work directory of the old eiger computer? Of course, I can move the old data folder to the new filesystem manually.
You can simply load a verdi shell and do the following:
load_computer('eiger').set_workdir('/new/path')
Then it would probably be a good idea to move existing data to this path as well, just so you can copy from RemoteData still.
Xing
October 30, 2023, 11:15am
3
Thanks @sphuber . Yes, set_workdir works, and I can use the remote_folder now.
There is one problem, that is not critical in my case, which is that verdi calcjob gotocomputer does not work for the previous process. I check the cmd, and it read the REMOTE_WORKDIR_KEY,
return self.base.attributes.get(self.REMOTE_WORKDIR_KEY, None)
which is hardcoded, and will not be updated with the computer’s workdir.
In [6]: n.base.attributes.all['remote_workdir']
Out[6]: '/scratch/e1000/xingwang/aiida/62/6a/985e-a6ed-45f4-bbeb-80cd99851143'
system
October 30, 2023, 3:16pm
4
This topic was automatically closed 4 hours after the last reply. New replies are no longer allowed.
mbercx
November 8, 2023, 8:43pm
6
Reopened this discussion, as I don’t think it’s really finished. ^^
I wasn’t aware we could change the workdir actually, so I’ve learned something new. Have added a comment to a closely related issue on the road map:
opened 04:33PM - 01 Mar 23 UTC
roadmap/proposed
## Motivation
When computers and codes were originally designed, a certain le… vel of rigidity on their properties was established in order to try to impose a stricter level of control over the preservation of the provenance. However, the frequent changes an updates that computer clusters perform on their systems forces users to be also updating the ways in which they access those resources. Part of this required versatility was taken into account by allowing some parameters of computer access to be set in the `configuration` of the computer, rather than in its attributes during setup. However, many users find that this is not enough and express frustration at the limitations that these entities still have.
Some examples include:
- After updating software in clusters, some software change location and thus the respective code needs to be updated or changed to a new one. Creating a new code is the natural solution, but it tends to then over-complicate querying later.
- One may want to set up a computer to be able to be used with different schedulers (specially considering the inclusion of meta-schedulers such as [aiida-hyperqueue](https://github.com/aiidateam/aiida-hyperqueue)), but by doing so the remote folders created by one are technically in a different computer and thus copying of files between those is not allowed.
## Desired Outcome
Rethink how immutability works in the computer and code nodes, what restrictions can be relaxed, which ones we want to keep to ensure a cleaner provenance, and what other options can we offer users to facilitate dealing with these immutability problems in the cases that need to remain.
## Impact
This will potentially improve usability for all users of AiiDA.
## Complexity
Since this is not a bug or missing feature, but a direct consequence of how we enforce the "respect" for the provenance and this clashing or resulting inconvenience in practical usage, the biggest issue here is to rethink a more general principle of AiiDA. Since this is coming from unforeseen practical inconvenience, it is worth it to first try to do a thorough compilation of user frustrations and also some thinking into other potential problems.
## Progress
The first task should be to do a comprehensive analysis of the situation and gather any remaining use case that may be relevant to consider.