Hello!
I was performing some migrations of some repositories by dumping the SQL database and copying the repo files. I noticed that for some instances the repository_metadata was lost which causes that I cannot access certain files. I was wondering if someone else had experienced something similar, and in that case what could be done. I can try doing the dumping again, but I just want to see if this is an isolated incident or something that we know can happen.
Thanks!
Cheers
Hi @JPchico , could you please provide some more information. What version of AiiDA are the original databases at? Did you verify that those actually contain the repository_metadata? If so, how did you actually dump and restore the data. Did you check the dump manually to see if the repository metadata is present? Are you using the same Python environment for the restored databases?
Hi, I actually very recently noticed something strange too. When exporting to a .aiida repository, some data in certain files is not preserved. This happened to some aiida.xml files. When importing to a new profile (different python environment), some of the xml keys were missing. Don’t know if it’s a similar issue.
Hi @sphuber, sure thing. What I did was move the repo and database to another server since the one we were using was too old and it was not possible to update anymore. The aiida version for the old repo was 1.6.9 the same than for the newer server. The older had python 3.8 in a virtual environment while the newest one does the same with python3.10. The servers changed from ubuntu 18.04 and the other one was ubuntu 22.04. After that I imported data from another profile that was running aiida-2.0.2 this to try to centralize some of the data.
The majority of the affected nodes are arrays, it seems that for some reason the repository_metadata for those was not copied? Or perhaps during the migration it disappeared. Funnily enough I have done the same procedure in another machine and in there the issue does not seem to happen?
If I run this in the affected installation I see a lot of arrays
from aiida import orm
qb = orm.QueryBuilder()
qb.append(orm.ArrayData)
affected = 0
healthy = 0
for node in qb.all(flat=True):
for _array in node.get_arraynames():
try:
temp = node.get_array(_array)
healthy +=1
except:
affected +=1
print(f'Affected arrays {affected}')
print(f'Healthy arrays {healthy}')
Affected arrays 1704
Healthy arrays 7503
I’m guessing that it is the migration the one causing this, I still do not understand why it worked in one system and not in another, I guess the order of operations was slightly different.
Hi @JPchico , the repository_metadata column was introduced with the new repository which was only added in v2.0. So your profiles at v1.6.9 would still have had the old repository structure with the many sharded subdirectories. So that excludes a problem in the dumping/restoring of the SQL database. The remaining options are that the copying of the repository was not complete. I take it that you are working with big profiles so the repository would have been quite large. Probably the transfer would have taken quite some time, even with rsync, right?
It would be good to get an ArrayNode that is affected and note its UUID and the name of the array that appears to be missing. Then you check in the repository on the original machine to see if the array is actually there. You can run verdi profile show to get the base path of the repository. If the UUID is then aabbcc-dddd-etc you should find the array file at {repo_basepath}/aa/bb/cc-dddd-etc/array_name.npy. It would be great if you can confirm that the missing arrays on the new machine are actually present in the old repo.
If that is the case, then it would be great if you could do the same check on the old repo on the new machine. It should have been migrated when you updated AiiDA to v2, however, the original would have been kept for safety purposes. Did you delete it manually afterwards? There should also have been log files with logging from the migration itself. Do you still have those files?
Hi! I think it is an incomplete transfer as you suspected, since if I look in the original files I can find the file in the path there, so probably something got loss when synching to the new machine (I have not been able to test all the nodes, since there are a lot of them, but I think it should be possible to recover most of them).
To do a proper restore, you have to do the following:
rsync ideally and verify the repo is fully copiedverdi storage migrate again. Since the old style repo should now be complete, it should now properly construct the repository_metadata.This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.