>>>>>>>> # Scatter wfc restart files<br>
awk '{ files_for[$1] = files_for[$1] " '$basename'.wfc" NR }<br>
END { for (host in files_for) print host, files_for[host] }' $PBS_NODEFILE \<br>
| while read host files<br>
do<br>
ssh -n $host "cd $PBS_O_WORKDIR; mv $files $ESPRESSO_TMPDIR/"<br>
done<br>
# on master host, copy .save directory as well<br>
rsync -a $basename.save $ESPRESSO_TMPDIR<br>
<br>
<br>
mpirun -x ESPRESSO_TMPDIR \<br>
-np $(wc -l < $PBS_NODEFILE) \<br>
-machinefile $PBS_NODEFILE \<br>
pw.x -inp input.txt > output.txt<br>
<br>
# Gather remote files<br>
uniq $PBS_NODEFILE \<br>
| while read host<br>
do<br>
ssh -n $host "rsync -a $TMPDIR/ $PBS_O_WORKDIR/"<br>
done<br>
------------------------------<br><div id=":13v">
<br>
E.g. for a job with nodes=3:ppn=4 the scatter part would distribute the existing files pwscf.wfc{1..12} as follows: <<<<<<<br><br>Yes, this looks indeed cumbersome. It becomes more painful when one can not know a priori in which nodes his/her job <br>
will go, particularly when it is totally decided by the automatic queue decider, depending on the free nodes available.<br>In such a situation, one's restarted job may go to a totally new set of nodes, and phonon calculation can not get necessary <br>
files to restart. Then restarting phonon calculation becomes more difficult. <br><br>It seems, there is a more serious trouble in the recent version of QE. In the version before QE4.2, the QE codes used to <br>replicate the same necessary files to the distributed local disks of all the nodes. In this case, at least phonon calculation <br>
can run smoothly instead of crashing. But in the recent version, phonon calculations just stop by complaining that <br>the distributed .wfc files in one node are not visible by another node. <br> <br>If a quick remedy of this problem is not easy, then at least, for the time being, it is better to keep the earlier option of replicating <br>
the same .wfc files in all the nodes still working in the version 4.3.1. Other better option can be to implement the "WF_COLLECT" <br>trick, also in phonon code, as it is already there for PW.x. <br><br>Thanks and regards, <br>
Saha SK<br>R&D Assistant <br>JNCASR <br>Bangalore 560064 <br></div><br><br>