Note: This discussion is about an older version of the COMSOL Multiphysics® software. The information provided may be out of date.
Discussion Closed This discussion was created more than 6 months ago and has been closed. To start a new discussion with a link back to this one, click here.
Parallel Computing with MATLAB Livelink and COMSOL
Posted 20 lug 2013, 13:37 GMT-4 Interfacing, LiveLink for MATLAB Version 4.3 12 Replies
Please login with a confirmed email address before reporting spam
Hi everyone,
I am currently trying to use Matlab's Global Optimization tool (specifically using genetic algorithms) with my function. Every time my function evaluates, it loads a model file, modifies some parameters in it, does the calculation and sends it back to the Matlab program. In order to speed up this process, I intend to parallelize my code. The Matlab part of this is easy, as I simply have to start a pool of MATLAB workers and turn the parallelization option in the optimization tool. However, for the COMSOL part, I think that I need an independent COMSOL server for every worker/core. How does one go about doing this?
I have tried the solution outlined in this thread (www.comsol.com/community/forums/general/thread/33446/). Specifically what I tried to do was write the following code to initialize multiple COMSOL servers:
matlabpool open 3
comsolPort = 2036;
system( ['comsol -np 1 server -silent -port ' num2str(comsolPort) ' &'] );
pause( 6 )
comsolPort = 2037;
system( ['comsol -np 1 server -silent -port ' num2str(comsolPort) ' &'] );
pause( 6 )
comsolPort = 2038;
system( ['comsol -np 1 server -silent -port ' num2str(comsolPort) ' &'] );
pause( 6 )
Then after this I start the Matlab genetic algorithm, which calls upon my function.
Within my function, I add an mphstart(portno), with the portno depending on the MATLAB worker ID.
What happens when I run this code is that at first three COMSOL servers seem to come up (with three splash screens etc.). Initially each worker seems to be able to start accessing the model. However after a few seconds (while the workers are modifying the model) the COMSOL servers shut down and only one worker is left doing its job. What is the cause of this? Is it that COMSOL doesn't allow more than one server operating?
On the other hand, when I tried to make all workers simply connect to the same COMSOL server, a "java.lang.NullPointerException" occurs, which led me to believe that I need three independent servers.
If anyone has any pointers on how to execute this task (running a parallel operation on MATLAB livelink where each worker has access to a COMSOL server, or perhaps do this without needing multiple COMSOL servers), that would be great!
Thanks in advance for any help given.
I am currently trying to use Matlab's Global Optimization tool (specifically using genetic algorithms) with my function. Every time my function evaluates, it loads a model file, modifies some parameters in it, does the calculation and sends it back to the Matlab program. In order to speed up this process, I intend to parallelize my code. The Matlab part of this is easy, as I simply have to start a pool of MATLAB workers and turn the parallelization option in the optimization tool. However, for the COMSOL part, I think that I need an independent COMSOL server for every worker/core. How does one go about doing this?
I have tried the solution outlined in this thread (www.comsol.com/community/forums/general/thread/33446/). Specifically what I tried to do was write the following code to initialize multiple COMSOL servers:
matlabpool open 3
comsolPort = 2036;
system( ['comsol -np 1 server -silent -port ' num2str(comsolPort) ' &'] );
pause( 6 )
comsolPort = 2037;
system( ['comsol -np 1 server -silent -port ' num2str(comsolPort) ' &'] );
pause( 6 )
comsolPort = 2038;
system( ['comsol -np 1 server -silent -port ' num2str(comsolPort) ' &'] );
pause( 6 )
Then after this I start the Matlab genetic algorithm, which calls upon my function.
Within my function, I add an mphstart(portno), with the portno depending on the MATLAB worker ID.
What happens when I run this code is that at first three COMSOL servers seem to come up (with three splash screens etc.). Initially each worker seems to be able to start accessing the model. However after a few seconds (while the workers are modifying the model) the COMSOL servers shut down and only one worker is left doing its job. What is the cause of this? Is it that COMSOL doesn't allow more than one server operating?
On the other hand, when I tried to make all workers simply connect to the same COMSOL server, a "java.lang.NullPointerException" occurs, which led me to believe that I need three independent servers.
If anyone has any pointers on how to execute this task (running a parallel operation on MATLAB livelink where each worker has access to a COMSOL server, or perhaps do this without needing multiple COMSOL servers), that would be great!
Thanks in advance for any help given.
12 Replies Last Post 3 dic 2016, 09:09 GMT-5
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
I do not completely understand what you are doing with MATLAB, but from what it sounds like, I believe you need to be thinking about the "parametric sweep" mode of parallelism in using COMSOL in parallel. In this mode, you only need a single instance of a server.
Also, you have not mentioned what type of COMSOL license, nor how many licenses of COMSOL you have. In order to run in distributed parallel mode, you need a floating network license. In trying to understand your description each server of COMSOL would need one network license. Unless you have N floating network licenses of COMSOL, you will likely run out of licenses very quickly.
Instead, you can use a SINGLE floating network license, and distribute N independent COMSOL jobs across the cluster you are using. Each COMSOL job is associated with a MATLAB "worker" that is coupled to MATLAB. You may need to insert your MATLAB function within the COMSOL parametric sweep loop; i.e., call MATLAB from COMSOL instead of vice versa.
Also, you have not mentioned what type of COMSOL license, nor how many licenses of COMSOL you have. In order to run in distributed parallel mode, you need a floating network license. In trying to understand your description each server of COMSOL would need one network license. Unless you have N floating network licenses of COMSOL, you will likely run out of licenses very quickly.
Instead, you can use a SINGLE floating network license, and distribute N independent COMSOL jobs across the cluster you are using. Each COMSOL job is associated with a MATLAB "worker" that is coupled to MATLAB. You may need to insert your MATLAB function within the COMSOL parametric sweep loop; i.e., call MATLAB from COMSOL instead of vice versa.
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Hi James,
Thanks for your reply. I am not sure that parametric sweep is what I want (although I am quite new to COMSOL so I am not entirely sure what it can do either). In addition, there are 4 independent parameters in my model, which results in a lot of combinations if we were to really sweep them all, if that's what you meant. What my MATLAB script does is feed a random combination of values for these 4 parameters to the COMSOL model. The COMSOL server calculates only one part (in this case, the resulting electric field). The 2D electric field is then fed into MATLAB again to be further processed. This process is repeated many times over a random series of combinations. (Ultimately the genetic algorithm will sort out the better combinations from the worse ones). Because the bulk of the calculation is done in MATLAB (specifically the optimizing and also the processing after I obtain the electric field every time), I think it would be easier to call COMSOL from MATLAB, instead of vice versa.
Because there are so many combinations, I want to parallelize the code - several MATLAB workers do different combinations at the same time. But these workers have to each access the COMSOL model on their own. When I just naively ask each worker to connect to the same port using mphstart, it only works for one worker. I certainly understand your point about exhausting the number of licenses - and my university only has a limited number of licenses. But how do I set up a single COMSOL server so that it is accessible to several MATLAB workers?
Again, thanks for your assistance.
Thanks for your reply. I am not sure that parametric sweep is what I want (although I am quite new to COMSOL so I am not entirely sure what it can do either). In addition, there are 4 independent parameters in my model, which results in a lot of combinations if we were to really sweep them all, if that's what you meant. What my MATLAB script does is feed a random combination of values for these 4 parameters to the COMSOL model. The COMSOL server calculates only one part (in this case, the resulting electric field). The 2D electric field is then fed into MATLAB again to be further processed. This process is repeated many times over a random series of combinations. (Ultimately the genetic algorithm will sort out the better combinations from the worse ones). Because the bulk of the calculation is done in MATLAB (specifically the optimizing and also the processing after I obtain the electric field every time), I think it would be easier to call COMSOL from MATLAB, instead of vice versa.
Because there are so many combinations, I want to parallelize the code - several MATLAB workers do different combinations at the same time. But these workers have to each access the COMSOL model on their own. When I just naively ask each worker to connect to the same port using mphstart, it only works for one worker. I certainly understand your point about exhausting the number of licenses - and my university only has a limited number of licenses. But how do I set up a single COMSOL server so that it is accessible to several MATLAB workers?
Again, thanks for your assistance.
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Hi Daniel,
I am in exactly the same situation as you. May I ask if you have solved the problem? Could you please kindly share your solution here?
Really appreciate your help!!!
Thanks a lot!
Best regards,
Alvin
I am in exactly the same situation as you. May I ask if you have solved the problem? Could you please kindly share your solution here?
Really appreciate your help!!!
Thanks a lot!
Best regards,
Alvin
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Hi Sir,
Could you please advise how you "can use a SINGLE floating network license, and distribute N independent COMSOL jobs across the cluster you are using"?
Thanks a lot!!
Best regards,
Alvin
Could you please advise how you "can use a SINGLE floating network license, and distribute N independent COMSOL jobs across the cluster you are using"?
Thanks a lot!!
Best regards,
Alvin
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
repeating exactly the same thing I said before, you use parametric sweeps
"Instead, you can use a SINGLE floating network license, and distribute N independent COMSOL jobs across the cluster you are using. Each COMSOL job is associated with a MATLAB "worker" that is coupled to MATLAB. You may need to insert your MATLAB function within the COMSOL parametric sweep loop; i.e., call MATLAB from COMSOL instead of vice versa."
"Instead, you can use a SINGLE floating network license, and distribute N independent COMSOL jobs across the cluster you are using. Each COMSOL job is associated with a MATLAB "worker" that is coupled to MATLAB. You may need to insert your MATLAB function within the COMSOL parametric sweep loop; i.e., call MATLAB from COMSOL instead of vice versa."
Please login with a confirmed email address before reporting spam
Posted:
1 decade ago
Hi Daniel,
I'm trying to use Comsol with the optimization tool from Matlab to estimate some parameters of my Comsol model using genetic algorithms. Could you please, give me some tips how to do it? I'm not trying to do parallel processing, it's just the first part of your text, which I think you are dealing well with that, right?
Could you please help me with that?
Thank you.
Marco
I'm trying to use Comsol with the optimization tool from Matlab to estimate some parameters of my Comsol model using genetic algorithms. Could you please, give me some tips how to do it? I'm not trying to do parallel processing, it's just the first part of your text, which I think you are dealing well with that, right?
Could you please help me with that?
Thank you.
Marco
Please login with a confirmed email address before reporting spam
Posted:
10 years ago
Hi all,
maybe it's a little late now.
Enclosed in this message, you can find the MATLAB code that allows to use Comsol with the optimization tool from Matlab (genetic algorithms) to estimate some parameters of my COMSOL model (my OS is Windows 7).
In the MATLAB file, @inverse2Dparga is my function that contains my COMSOL model. Following the first lines of my MATLAB function @inverse2Dparga:
function J=inverse2Dparga(b)
cd('D:\COMSOL43b\mli')
comsolPort=[2036 2037];
t = getCurrentTask();
labit=t.ID;
mphstart(comsolPort(labit))
import('com.comsol.model.*');
import('com.comsol.model.util.*');
...
(here, b is my parameter vector generated by MATLAB optimization algorithm and used in COMSOL;
J is the objective function from COMSOL).
I precise that these code lines allow to perform parallel computations and it works fine.
Kind regards,
DG.
maybe it's a little late now.
Enclosed in this message, you can find the MATLAB code that allows to use Comsol with the optimization tool from Matlab (genetic algorithms) to estimate some parameters of my COMSOL model (my OS is Windows 7).
In the MATLAB file, @inverse2Dparga is my function that contains my COMSOL model. Following the first lines of my MATLAB function @inverse2Dparga:
function J=inverse2Dparga(b)
cd('D:\COMSOL43b\mli')
comsolPort=[2036 2037];
t = getCurrentTask();
labit=t.ID;
mphstart(comsolPort(labit))
import('com.comsol.model.*');
import('com.comsol.model.util.*');
...
(here, b is my parameter vector generated by MATLAB optimization algorithm and used in COMSOL;
J is the objective function from COMSOL).
I precise that these code lines allow to perform parallel computations and it works fine.
Kind regards,
DG.
Attachments:
Please login with a confirmed email address before reporting spam
Posted:
9 years ago
Dear James,
Similar to the original case mentioned at the beginning of this trend, for me it is not possible to call Matlab functions from Comsol to run parametric sweep loop. Different to the original case of a single model being solved several times, in my case the generated models from Matlab are different, and the calculation times are growing to the infinity and beyond. That is why I have the chance to use at least two or three floating network licenses. But again I'm facing the same problem:
***************
Error using mphstart (line 152)
A connection to Comsol could not be established. Please check that a Comsol Server is started prior to calling mphstart
****************
I've done my homework and tried most of the solutions proposed in the comsol forums, even if my version is COMSOL 4.4 (Build: 248).
Is there any possible trick, solution or advice that could help me speeding up my calculation times?
Thnx!
Similar to the original case mentioned at the beginning of this trend, for me it is not possible to call Matlab functions from Comsol to run parametric sweep loop. Different to the original case of a single model being solved several times, in my case the generated models from Matlab are different, and the calculation times are growing to the infinity and beyond. That is why I have the chance to use at least two or three floating network licenses. But again I'm facing the same problem:
***************
Error using mphstart (line 152)
A connection to Comsol could not be established. Please check that a Comsol Server is started prior to calling mphstart
****************
I've done my homework and tried most of the solutions proposed in the comsol forums, even if my version is COMSOL 4.4 (Build: 248).
Is there any possible trick, solution or advice that could help me speeding up my calculation times?
Thnx!
Please login with a confirmed email address before reporting spam
Posted:
9 years ago
Can you set up your problem to run without MATLAB just to test it ? I suspect something is not correct about your MATLAB to COMSOL setup. COMSOL tech support can help you fix this.
Please login with a confirmed email address before reporting spam
Posted:
9 years ago
Hi all,
maybe it's a little late now.
Enclosed in this message, you can find the MATLAB code that allows to use Comsol with the optimization tool from Matlab (genetic algorithms) to estimate some parameters of my COMSOL model (my OS is Windows 7).
In the MATLAB file, @inverse2Dparga is my function that contains my COMSOL model. Following the first lines of my MATLAB function @inverse2Dparga:
function J=inverse2Dparga(b)
cd('D:\COMSOL43b\mli')
comsolPort=[2036 2037];
t = getCurrentTask();
labit=t.ID;
mphstart(comsolPort(labit))
import('com.comsol.model.*');
import('com.comsol.model.util.*');
...
(here, b is my parameter vector generated by MATLAB optimization algorithm and used in COMSOL;
J is the objective function from COMSOL).
I precise that these code lines allow to perform parallel computations and it works fine.
Kind regards,
DG.
Hi DG,
I am also running optimization programs with comsol server in Matlab. Do you have the problem with memory (RAM) accumulation when the optimization keeps going on and is generating more individuals? Such accumulation of RAM usage causes the RAM to run out after a small number of generations. This problem is caused in the comsol server which is not able to clean up the RAM after one generation of optimization. I do not know if you have any solutions to deal with this. What I am doing is to close the comsol server after each generation and then restart it using some codes in Matlab. This closes the chance to run several comsol servers at the same time, and then does not allow parallel computing in multiple comsol servers. I really need a method to clean up the RAM use or in the comsol server after each generation or reinitialize the comsol server without closing and restarting it.
Thanks a lot!
Best regards,
Dong
Please login with a confirmed email address before reporting spam
Posted:
9 years ago
Hello DG and COMSOL users,
Its a late message,
Your code snippet to use Comsol with the optimization tool from Matlab is really useful. However, i am facing a small issue.
At the function @inverse2Dparga if i use the below steps as it is:
function J=inverse2Dparga(b)
cd('D:\COMSOL43b\mli')
comsolPort=[2036 2037];
t = getCurrentTask();
labit=t.ID;
mphstart(comsolPort(labit))
import('com.comsol.model.*');
import('com.comsol.model.util.*');
I get the below error:
"Attempt to reference field of non-structure array" @ step "labit=t.ID". I am using COMSOL version 5.0.
However if i use this code inside parfor loop it works. Do i need to wrap this function in a 'parfor' loop?
Is there any ways to do it without using parfor loop?
After doing this i am loading my model using
model = mphload('LIB_9AH_1D.mph');
This model gets loaded each time when this function is called [I am just sweeping some parameter values in the model] which will bring down its permformance.
Is there any ways to do the same operation in the main function?
Any ideas will really help to continue with my work.
Thank you.
yours sincerley,
Chris
Its a late message,
Your code snippet to use Comsol with the optimization tool from Matlab is really useful. However, i am facing a small issue.
At the function @inverse2Dparga if i use the below steps as it is:
function J=inverse2Dparga(b)
cd('D:\COMSOL43b\mli')
comsolPort=[2036 2037];
t = getCurrentTask();
labit=t.ID;
mphstart(comsolPort(labit))
import('com.comsol.model.*');
import('com.comsol.model.util.*');
I get the below error:
"Attempt to reference field of non-structure array" @ step "labit=t.ID". I am using COMSOL version 5.0.
However if i use this code inside parfor loop it works. Do i need to wrap this function in a 'parfor' loop?
Is there any ways to do it without using parfor loop?
After doing this i am loading my model using
model = mphload('LIB_9AH_1D.mph');
This model gets loaded each time when this function is called [I am just sweeping some parameter values in the model] which will bring down its permformance.
Is there any ways to do the same operation in the main function?
Any ideas will really help to continue with my work.
Thank you.
yours sincerley,
Chris
Please login with a confirmed email address before reporting spam
Posted:
8 years ago
Hi DG
Thank you very much for your code. Your code is really useful to paralle computing by Comsol with matlab. However, I am facing a small issue. Below is my code. Don't know why the code runs only once for parallel loop (only execute two step of loop simultaneously), the next step occurs an error. (Comsol 4.3, MATLA R2010b, win7 flagship version)
??? Error using ==> parallel_function at 598
Error in ==> parallel_function>make_general_channel/channel_general at 879
Undefined function or method 'optimization' for input arguments of type 'double'.
Error in ==> test at 30
%%%%%%%%%%%%%%%
matlabpool open 2;
comsolPort1 = 2036;
system( ['comsolserver -np 1 -port' num2str(comsolPort1) ' &'] );
pause(1)
comsolPort2 = 2037;
system( ['comsolserver -np 1 -port ' num2str(comsolPort2) ' &'] );
pause(1)
parfor i=1:NP
fitnessX(i)=optimization(X(i,:));%fitnessX表示X的适应值
end
matlabpool close;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%55
function Q = optimization(X)
cd('D:\Program Files\MATLAB\R2010b\COMSOL43\mli')
comsolPort=[2036,2037];
t = getCurrentTask();
labit=t.ID;
mphstart(comsolPort(labit));
import com.comsol.model.*
import com.comsol.model.util.*
model = ModelUtil.create('Model');
....................
Thank you a lot!
Regards
Jia
Thank you very much for your code. Your code is really useful to paralle computing by Comsol with matlab. However, I am facing a small issue. Below is my code. Don't know why the code runs only once for parallel loop (only execute two step of loop simultaneously), the next step occurs an error. (Comsol 4.3, MATLA R2010b, win7 flagship version)
??? Error using ==> parallel_function at 598
Error in ==> parallel_function>make_general_channel/channel_general at 879
Undefined function or method 'optimization' for input arguments of type 'double'.
Error in ==> test at 30
%%%%%%%%%%%%%%%
matlabpool open 2;
comsolPort1 = 2036;
system( ['comsolserver -np 1 -port' num2str(comsolPort1) ' &'] );
pause(1)
comsolPort2 = 2037;
system( ['comsolserver -np 1 -port ' num2str(comsolPort2) ' &'] );
pause(1)
parfor i=1:NP
fitnessX(i)=optimization(X(i,:));%fitnessX表示X的适应值
end
matlabpool close;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%55
function Q = optimization(X)
cd('D:\Program Files\MATLAB\R2010b\COMSOL43\mli')
comsolPort=[2036,2037];
t = getCurrentTask();
labit=t.ID;
mphstart(comsolPort(labit));
import com.comsol.model.*
import com.comsol.model.util.*
model = ModelUtil.create('Model');
....................
Thank you a lot!
Regards
Jia