druck  writes:
>On 12/07/2018 18:27, Anton Ertl wrote:
>> You have to be yet clearer.  This claim is still nonsense.  If you
>> neglect individual CPU cycles such that each individual CPU takes,
>> say, twice as many cycles the whole application will take twice as
>> long, on the same parallel machine.
>No it wont.
>
>> The parallel machine guys buy
>> these expensive parallel boxes so that their applications run faster,
>> so they certainly would not appreciate this kind of waste.
>
>Perhaps they know more about them than you do.

They sure do.  But they are not making the unsubstantiated claim, john
is.

>Massively parallel systems have a huge theoretical performance from
>thousands interconnected systems each with their own CPUs and memory.
>However any real computational problem can only use a faction of that
>theoretical performance due to the issue of transferring data over the
>slow interconnects.
>
>To be able to scale a problem to work efficiently on an MPP system, the
>most important issue is to work out how to partition the data and code
>the algorithm to minimise the need to transfer data across the
>interconnects. This can lead to code being very different and much
>slower than that which would be use on a conventional single unified
>memory machine.
>
>To take your example, the code on each machine might take twice a many
>individual cycles as it could do, but if it avoids even a small amoint
>of inter machine data transfer, it may run many hundreds of times faster
>on a massively parallel system.

Well, at least that's something more substantial.  There are some
holes in the argument, though:

Even if an application is completely communications-bound, it needs to
improve communications overhead by hundreds of times (not just a small
amount) to make it run hundreds of times faster.

The main question, though, is: If, as you claim, communication rather
than computation was the major contributor to run-time in parallel
applications, why are they so stupid and run the application on so many
CPUs?  It would be more efficient to run it on fewer CPUs, with less
communications overhead.  And once you are there, CPU cycles matter
again.

Still, if the claim was that they invest CPU cycles in reorganizing
the data such that they can distribute the application to more CPUs
without running into communications bottlenecks, yes, I can believe
that.  But that's not what john claimed.

- anton
--
M. Anton Ertl  http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
     New standard: http://www.forth200x.org/forth200x.html
   EuroForth 2018: http://www.euroforth.org/ef18/

--- SoupGate-Win32 v1.05