[Pw_forum] How to using gamma point calculation with high efficiency

[Axel Kohlmeyer]

On Sun, 29 Jun 2008, vega lew wrote:

VL> 
VL> > Does this mean, that you have single socket quad core CPU/machines, and you 
VL> > have 5 nodes in your cluster? What is the interconnect between the machines?
VL> 
VL> Yes, my mashines is a single socket on system board. My cpu is intel Core TM II Q6600.
VL> The network between my cluster is only 1Gbit ethernet using TCP/IP.

just for the sake of completeness. it is not the bandwidth of GigE
networking that is limiting, but the overhead of TCP/IP networking.
for g-space parallelization you have a lot of messages (k-point
parallelization is more of the embarrassingly parallel type of 
problem) and the high latency time of sending TCP/IP encoded data
is killing the performance. you would see a similar behavior with
with running TCP/IP over myrinet or infiniband. OTOH, there are
attempts to implement MPI libraries that use GigE hardware, but
bypass the TCP/IP encoding (you need a second set of network card 
and a second switch for that), but those are generally only working
for a specific hardware/software combination due to lack of manpower.
none of the (big) companies is supporting those efforts, since that
would impact their business of selling you the more expensive 
high performance hardware. as in (interesting?) aside. the myrinet
MX hardware is actually 10Gigabit ethernet. the magic is all in
the software... ;-).

[...]

VL> My most interested system is the Titanium Oxide surface with 1~3 
VL> noble metal atom (too many is more interested, but DFT can't handle 

what do you mean by DFT cannot handle this? 
you should have learned by now that making general statements
like this without any backing can be very "dangerous". ;-)

[...]

VL> How could I exploit the parallelization over G-space? Only by use 
VL> the the proper number of CPU? no pools can be used?

please _read_ the documentation that you have been asked to read
now several times. g-space parallelization is the default for pw.x.
you don't have to turn it on. 

VL> What's the exact meaning of Gigabit LAN is just too slow for that
VL> (except for insanely large supercells)? Do you mean I could 
VL> calculate with relative high efficiency by using large systems? How 
VL> many atoms do you call it 'large' supercells?

before discussing any of these issues any further, you should inform 
yourself about parallel efficiency in particular and parallelization in
general. it is pretty pointless to discuss details, if you don't know
the basics well. the following is a "for dummies"-style description
that will hopefully get you started looking into the right direction.

TCP/IP networking incurs communication latencies. the g-space parallelization 
distributes data across the MPI tasks, i.e. the more nodes you use, the 
less work you have to do on each node. at the same time the average wait 
time for a parallel message across the machine goes up (since you have 
to contact more nodes). when this wait time is more than the time spent 
on processing the data, you have reached the limit of scaling. 
if you make your problem bigger (more data), you reach the limit of 
scaling later, but since the physics of the problem does not scale
linearly with the problem set size, your calculation goes in total
slower. you only waste less time.

in summary, if your hardware doesn't support good scaling, you don't
get it. at least not unless you invent a new algorithm that needs 
less communication or can hide the communication latencies. this is 
the reason why you have to make some benchmarks first to evaluate the
scaling for your problem on your machine.

cheers,
   axel.

-- 
=======================================================================
Axel Kohlmeyer   akohlmey at cmm.chem.upenn.edu   http://www.cmm.upenn.edu
   Center for Molecular Modeling   --   University of Pennsylvania
Department of Chemistry, 231 S.34th Street, Philadelphia, PA 19104-6323
tel: 1-215-898-1582,  fax: 1-215-573-6233,  office-tel: 1-215-898-5425
=======================================================================
If you make something idiot-proof, the universe creates a better idiot.