Discussion Closed This discussion was created more than 6 months ago and has been closed. To start a new discussion with a link back to this one, click here.
How to properly export and import data?
Posted 3 apr 2017, 03:49 GMT-4 3 Replies
Please login with a confirmed email address before reporting spam
Hi, I have two models where the second one uses results of the first one.
I export the results of model one to a spreadsheet of 4000x4000 points (regular grid), corresponding to a resolution of 0.5 nm. This results in a huge csv-file of almost 1 GB. I then import and plot this file into the model two via an interpolation function, but this can take up to 15 minutes. This is all done on a dedicated Linux server with 128 GB memory, which is normally very fast.
Is there a better way for me to export/import data between models?
I export the results of model one to a spreadsheet of 4000x4000 points (regular grid), corresponding to a resolution of 0.5 nm. This results in a huge csv-file of almost 1 GB. I then import and plot this file into the model two via an interpolation function, but this can take up to 15 minutes. This is all done on a dedicated Linux server with 128 GB memory, which is normally very fast.
Is there a better way for me to export/import data between models?
3 Replies Last Post 13 apr 2017, 04:05 GMT-4
Please login with a confirmed email address before reporting spam
Posted:
8 years ago
I export the results of model one to a spreadsheet of 4000x4000 points (regular grid)
...
Is there a better way for me to export/import data between models?
What data format are you using - Spreadsheet or Grid? The Spreadsheet format gives a file that is easy to process in external software, but the Grid data format gives a smaller file which is processed much faster when you use it in an interpolation function in COMSOL Multiphysics.
Please login with a confirmed email address before reporting spam
Posted:
8 years ago
Hi, I have two models where the second one uses results of the first one.
I export the results of model one to a spreadsheet of 4000x4000 points (regular grid), corresponding to a resolution of 0.5 nm. This results in a huge csv-file of almost 1 GB. I then import and plot this file into the model two via an interpolation function, but this can take up to 15 minutes. This is all done on a dedicated Linux server with 128 GB memory, which is normally very fast.
Is there a better way for me to export/import data between models?
I think that if it's absolutely necessary for you to have that kind of resolution then there probably isn't a way around how long it's going to take for the import. You can always specify the step-size of your grid and cut down the number of data points. Also, when you say that you take data from one model and import it in to another are you talking about two separate model files or two components?
Edit: I wrote my response before reading Gunnar's comment - I have only ever imported data using the Grid format and although I don't have 4000X4000 data points, I can tell you that importing with something around 500x500 in grid format is almost instant on a machine with 32 GB memory.
Please login with a confirmed email address before reporting spam
Posted:
8 years ago
I export the results of model one to a spreadsheet of 4000x4000 points (regular grid)
...
Is there a better way for me to export/import data between models?
What data format are you using - Spreadsheet or Grid? The Spreadsheet format gives a file that is easy to process in external software, but the Grid data format gives a smaller file which is processed much faster when you use it in an interpolation function in COMSOL Multiphysics.
Thanks, that did the trick! I was using Spreadsheet before, but Grid is significantly faster to the point that loading and plotting is almost instant.