<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div>I was able to make a bit more progress - at least in the direction of seeing what is breaking as I get to larger numbers of electrons. It seems to be mainly due to a number of large allocates which all of the processes carry out. I don't yet know enough about the code to know, but I suspect these aren't all necessary.</div><div><br></div><div>I have found 2 that have caused problems in different runs. The first was the problem that Axel pointed out earlier in this thread, the next is this one:</div><div><br></div><div>In my periodic bulk case (1156 atoms, 2560 electrons), running on 1024 procs in vn mode (256 nodes, 512 MB RAM/process):</div><div>"add_vuspsi.f90", line 78: 1525-108 Error encountered while attempting to allocate a data object. The program will stop.</div><div><br></div><div>- at this line, there is an allocate that is the size of: (number of projectors for the atom types) x (number of states).... (so something like 4736 * 1536 ... ~55 MB) which seems to be kicking it over the limit. This prevented the system from getting to where it would ouput info about the SCF steps.</div><div><br></div><div>My memory report output for this run looks like:</div><div><br></div><div><div>Largest allocated arrays est. size (Mb) dimensions</div><div> Kohn-Sham Wavefunctions 3.35 Mb ( 143,1536)</div><div> NL pseudopotentials 10.33 Mb ( 143,4736)</div><div> Each V/rho on FFT grid 0.78 Mb ( 51200)</div><div> Each G-vector array 0.01 Mb ( 1006)</div><div> G-vector shells 0.00 Mb ( 488)</div><div> Largest temporary arrays est. size (Mb) dimensions</div><div> Auxiliary wavefunctions 6.70 Mb ( 143,6144)</div><div> Each subspace H/S matrix 288.00 Mb ( 6144,6144)</div><div> Each <psi_i|beta_j> matrix 55.50 Mb ( 4736,1536)</div><div> Arrays for rho mixing 6.25 Mb ( 51200, 8)</div><div><br></div><div>So I am guessing that something just isn't getting split up right or at least not very efficiently - probably the Hamiltonian and Overlap matrices as a start.</div></div><div><br></div><div><br></div><div>The above case in dual mode (512 nodes, 1 GB RAM/process) was able to get into the SCF stages, and do some output:</div><div><br></div><div><div>Self-consistent Calculation</div><div><br></div><div> iteration # 1 ecut= 38.22 Ry beta=0.70</div><div> Davidson diagonalization with overlap</div><div><br></div><div>but then died with the following memory-related error:</div><div><br></div><div>"regterg.f90", line 108: 1525-108 Error encountered while attempting to allocate a data object. The program will stop.</div><div><br></div><div>Which appears to be caused by an allocation that looks like the # planewaves * # states (or some subset of them).</div><div><br></div><div>I guess this isn't really a surprise to you guys, and I am not really sure what to do about it, but at least I am now getting some idea of what is causing the breakdown.</div><div><br></div><div>Dave</div></div><div><br></div><div><br></div><div><br></div><br><div><div>On Feb 11, 2009, at 7:18 AM, Paolo Giannozzi wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>Hi, any news on your BG problem? Paolo<br><br>-- <br>Paolo Giannozzi, Democritos and University of Udine, Italy<br></div></blockquote></div><br><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; font-size: 12px; ">David E. Farrell</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size: 12px; ">Post-Doctoral Fellow</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size: 12px; ">Department of Materials Science and Engineering</span></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; font-size: 12px; ">Northwestern University</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: normal normal normal 12px/normal Helvetica; font-size: 12px; ">email: <a href="mailto:d-farrell2@northwestern.edu">d-farrell2@northwestern.edu</a></font></div></div></div></span> </div><br></body></html>