[Pw_forum] scaling on clusters with different communication types

[Kara, Abdelkader]

Thanks Axel for your valuable reply.
I am actually looking for a quantitative comparison.
I want to know if by using the infiniband or myrinet one can push the good (quasilinear)
scaling to a higher number of CPU's. If someone did any benchmarking, I will
appreciate your input.
 
Thanks again
 
Kader Kara
UCF

________________________________

From: Axel Kohlmeyer [mailto:akohlmey at cmm.chem.upenn.edu]
Sent: Fri 12/8/2006 8:47 PM
To: Kara, Abdelkader
Cc: pw_forum at pwscf.org
Subject: Re: [Pw_forum] scaling on clusters with different communication types



On Fri, 8 Dec 2006, Kara, Abdelkader wrote:

AK> Dear all,
AK>
AK> Greatings.
AK>
AK> I will appreciate it very much if you can share with me your experience
AK> of running pwscf on clusters with different communication hardware.
AK> I am interested in the scaling with the number of CPU's for the following 3
AK> different communication types:
AK> 1)gigabit ethernet
AK> 2) myrinet
AK> 3) InfiniBand

scaling depends a lot on the kind of jobs you intend to run.

pw.x scales almost independendly and very well across NEB
images and k-points even with gigabit ethernet. on top of
that you can parallelize across g-space, which is much more
demanding in terms of communication bandwidth and latency.
in this case scaling across gigabit is limited to a few
nodes. in-node performance is governed by available memory
bandwidth wich results in hyper-threading being conter-productive,
multi-core cpus having reduced efficiency (depending on job
size, i.e. cache efficiency) and opteron cpus due to dedicated
per-cpu memory busses scaling better than intel (xeon). only
very recent intel (woodcrest) xeon cpus have been demonstrated
to have a somewhat better performance and price/performance ratio.

please note, that these are some general trends observed from
some usage patterns that may not translate to your needs.
also presence of a per-node local scratch area or absence
impacts the performance. using a NFS filesystem for temporary
storage usually results in degraded performance.

basically, the larger your systems and the fewer k-points
you need use, the more important a fast interconnect becomes.
performance between infiniband and myrinet solutions is
more or less equivalent when compared with gigabit. thus
using older/obsolete hardware can be a bargain.

cheers,
     axel.

AK>
AK> Thank you very much for your input on this matter
AK>
AK> Kader Kara
AK>
AK> Physics Department
AK> University of Central Florida
AK> _______________________________________________
AK> Pw_forum mailing list
AK> Pw_forum at pwscf.org
AK> http://www.democritos.it/mailman/listinfo/pw_forum
AK>

--
=======================================================================
Axel Kohlmeyer   akohlmey at cmm.chem.upenn.edu   http://www.cmm.upenn.edu
   Center for Molecular Modeling   --   University of Pennsylvania
Department of Chemistry, 231 S.34th Street, Philadelphia, PA 19104-6323
tel: 1-215-898-1582,  fax: 1-215-573-6233,  office-tel: 1-215-898-5425
=======================================================================
If you make something idiot-proof, the universe creates a better idiot.