Edgar J. Kaiser
Certified Consultant
Please login with a confirmed email address before reporting spam
Posted:
5 years ago
23 gen 2020, 08:27 GMT-5
Eman,
this is an interesting question I had been struggling with as well. While parametric sweeps may run independently for each parameter value and can thus be efficiently distributed on cluster nodes this is not the case for the time steps in a transient study.
So as long as your model fits into one node memory wise, the cluster will probably add overhead.
Now, I have to admit that I don't have a cluster available and so I can't test and challenge this hypothesis. Would be interesting to hear other's experiences and opinions.
Cheers
Edgar
-------------------
Edgar J. Kaiser
emPhys Physical Technology
www.emphys.com
Eman,
this is an interesting question I had been struggling with as well. While parametric sweeps may run independently for each parameter value and can thus be efficiently distributed on cluster nodes this is not the case for the time steps in a transient study.
So as long as your model fits into one node memory wise, the cluster will probably add overhead.
Now, I have to admit that I don't have a cluster available and so I can't test and challenge this hypothesis. Would be interesting to hear other's experiences and opinions.
Cheers
Edgar
Jim Freels
mechanical side of nuclear engineering, multiphysics analysis, COMSOL specialist
Please login with a confirmed email address before reporting spam
Posted:
5 years ago
23 gen 2020, 17:06 GMT-5
Given a problem large enough to benefit from cluster computing, then the benefit for time-dependent study is gained by the benefit of each separate time step. (This is essentially the same operation that takes place at end of each iteration of a steady-state problem, BTW.) COMSOL will need to gather up all the results from the split problem over the cluster nodes at the end of each iteration/time step before proceeding to split the work back up to the next iteration/time step.
A more interesting question to me is how many nodes can COMSOL effectively use on a large problem before the overhead required to scatter-gather the data overcomes the benefit in the work splitting of the problem over the nodes ? I still have not seen a study done by a COMSOL user that shows this. In my pre-retirement days, we were limited to 16 nodes; each node composed of large memory and processing nodes. There was no indication of slowdown in using all 16 nodes. The question is what about 100, 1000, or more nodes ?
In the paper/presentation linked below, presented at the 2010 COMSOL Conference in Boston, I showed representative speedup for COMSOL-4.0 when distributed parallel processing was first released. There have been many improvements in the code since that time, so the speedup should be much better now in the present version 5.5.
https://www.comsol.com/paper/exploiting-new-features-of-comsol-version-4-on-conjugate-heat-transfer-problems-7970
-------------------
James D. Freels, Ph.D., P.E.
Given a problem large enough to benefit from cluster computing, then the benefit for time-dependent study is gained by the benefit of each separate time step. (This is essentially the same operation that takes place at end of each iteration of a steady-state problem, BTW.) COMSOL will need to gather up all the results from the split problem over the cluster nodes at the end of each iteration/time step before proceeding to split the work back up to the next iteration/time step.
A more interesting question to me is how many nodes can COMSOL effectively use on a large problem before the overhead required to scatter-gather the data overcomes the benefit in the work splitting of the problem over the nodes ? I still have not seen a study done by a COMSOL user that shows this. In my pre-retirement days, we were limited to 16 nodes; each node composed of large memory and processing nodes. There was no indication of slowdown in using all 16 nodes. The question is what about 100, 1000, or more nodes ?
In the paper/presentation linked below, presented at the 2010 COMSOL Conference in Boston, I showed representative speedup for COMSOL-4.0 when distributed parallel processing was first released. There have been many improvements in the code since that time, so the speedup should be much better now in the present version 5.5.
https://www.comsol.com/paper/exploiting-new-features-of-comsol-version-4-on-conjugate-heat-transfer-problems-7970