`verdi archive` runs out of space even though it is available. Maybe chunking does not work?

General Usage
6

most probably the code is creating some temporary files on disk (maybe here? Or maybe somewhere else as well). This will go by default in the location specified in env vars like TMPDIR, TEMP or TMP (see more specific logic here). Typically it’s /tmp that is in a different partition.

Yes seems more promising. Will try it out

Just for completeness, what is the exact command that you invoked? And what is the output of df -h of the filesystem of the output file.

Filesystem on HPC and verdi archive commend

Right. So here again the relevant information of df -h and the whole log at the end

df: /scratch: Stale file handle
Filesystem                         Size  Used Avail Use% Mounted on
...
prnmarvelsrv4:/hith                 18T  5.2T   12T  32% /hith_nfs/prnmarvelsrv4 <--- where to archive
...
prnmarvelcommonfs:/home            3.7T  1.9T  1.6T  55% /home  <-- repository and $AIIDA_PATH

Command to archive

We used this command

verdi archive create -a /hith_nfs/prnmarvelsrv4/paulish_qiao-main.aiidaarchive

We checked that there is only one profile, so we don’t back up potentially files from other profiles with -a

AiiDA database information

How did you determine this? When you say database, do you mean the total storage, or just the PostgreSQL database? When you create an archive your are writing the sum of the PSQL database and the file repository to a single file. So your target diskspace should have at least that.

I meant the repository mainly. IT the the same as here Performance issues with verdi commands `verdi process` and `verdi storage` I will copy there things over

$ du -h --max-depth=1 /home/aiida/repository/main/container
4.0K	./duplicates
4.0K	./sandbox
2.0T	./loose
4.0K	./packs
2.0T	.

additonal information. Size of AIIDA path that keeps all the process id’s

$ du -h --max-depth=1 $AIIDA_PATH/.aiida/access
2.2G	/home/aiida/envs/aiida-fermisurf/.aiida/access/main
2.2G	/home/aiida/envs/aiida-fermisurf/.aiida/access

Number of files in this path

$ find $AIIDA_PATH/.aiida/access -type f | wc -l
557638

$ verdi storage info
entities:
  Users:
    count: 10
  Computers:
    count: 31
  Nodes:
    count: 4734923
  Groups:
    count: 331
  Comments:
    count: 0
  Logs:
    count: 1203422
  Links:
    count: 8936853
repository:
  SHA-hash algorithm: sha256
  Compression algorithm: zlib+1

Appendix

df_h.txt (1.8 KB)