formating csv file for quick access of list of integers
I want to read in lists of integers in OML. I figured that I might use a csv-file for it.
The data is organized in set ids and contains a varying number of integers, for exampe a set id 100 could be
set 100 = 1 20 30 44 500 1234
and maybe set 556 could be
set 556 = 12000 33456 120000 140000 160000
I am still rather free how the data is written into csv-file as I am writing from a Tcl-routine running in Hyperview/Hypermesh, so I could be like:
100 556
1 12000
20 33456
30 120000
44 140000
500 160000
1234
first row would be defining the set id and the rows below in the same column define then the ids belonging into the set. However this is a little bit tedious to write from tcl, it would be easier if I transposed it, but the problem might be then the line length as the number of ids can be rather high. I could write one set
over multiple lines:
| 100 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | |
| 100 | 9 | 10 | 11 | 12 | 13 | ||||
| 200 | 22 | 25 | 26 | 27 | |||||
| 300 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 |
| 300 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | |
| 300 | 47 | 48 | 49 | 50 |
Here set id 100 would be over two lines, set 200 needs only one line and 300 would be over 3 lines. I would then only need to concate the lists together, but I am not sure how to accomplish that in oml.
Can somebody give me a tip for a handy one-liner, I tend to think the last csv-format would be easiest to write from tcl but how do I combine the id lists from successive lines, in the end I might prefer to have something like:
set(100) = [1;2;3;4;5;6;7;8;9;10;11;12;13]
Find more posts tagged with
Altair EmployeeOk, two answer at least part of my own question - it is not so simple to find examples for oml.
Anyway if I went for the last format, repeating the set id always on column 1. I would get there
with:
setdata=csvread('sets.csv'); tmpsetdata=setdata(setdata(:,1) == 100,:); %gets matrix part where the first column is 100 (set id = 100) tmpsetdata(:,1) = 0; %sets the first column to 0 tmpsetdata=reshape(tmpsetdata,[],1); %reshape to 1 x [] matrix finalset=tmpsetdata(tmpsetdata(:,1) > 0,:); %omit the 0 valuesso, at least it is a five liner for starters.
Hi @Jyrki Majamaki,
Let me understand: Will you generate one CSV using TCL and want to use Compose to read this CSV and have the values of each set into arrays?
The OML provided by you already work for the format having the sets in multiple lines
What is your question now?
Hi Jyrki,
As Joao said, the commands you're using are correct, so I suppose you want to generalize it for sets that span multiple lines without explicitly declaring the set ID on each line, as we discussed previously?
In that case your csv file contains each set ID once, like this:
100,1,2,3,4,5,6,7,8
,9,10,11,12,13
200,22,25,26,27
300,30,31,32,33,34,35,36,37,38
,39,40,41,42,43,44,45,46
,47,48,49,50
And you could use something like this:
% Read data from CSV
filename = '.\sets.csv';
data = csvread(filename, 'emptyvalue', NaN);
% Get all sets (remove NaN)
setIDs = unique(data(:,1));
setIDs = setIDs(~isnan(setIDs));
% Get matrix of set definitions' locations
setDefs = data(:,1) == setIDs';
% Get number of sets and initialize cell for storing contents
nSets = size(setDefs,2);
sets = cell(nSets,1);
% Get elements for each set
for i = 1:nSets
% Get current set definition start
startIdx = find(setDefs(:,i));
% Get current set definition end
if i < nSets
endIdx = find(setDefs(:,i+1))-1;
else
endIdx = size(setDefs,1);
end
% Reshape set definition into single row and remove NaN
thisSet = reshape(data(startIdx:endIdx,:), 1, []);
sets{i} = thisSet(~isnan(thisSet));
end
Where sets.csv is the file you created in tcl.
Hope this works as a good indexing example as well.
Rafael
I realized that I have quite a bit of problem with my approach. Using the batchprocess of Hyperview, takes quite a bit of time and the problem is that the oml bit moves on. If I start the batch process, is there a way to make oml wait for the result? I could delete that transfer csvfile and make a while loop in oml to check against the existence of that file, but that is also a bit uncertain and has a taste of infinite loop.
I realized that I have quite a bit of problem with my approach. Using the batchprocess of Hyperview, takes quite a bit of time and the problem is that the oml bit moves on. If I start the batch process, is there a way to make oml wait for the result? I could delete that transfer csvfile and make a while loop in oml to check against the existence of that file, but that is also a bit uncertain and has a taste of infinite loop.
Hi Jyrki,
Yes, you could go on a loop and constantly check the file's existence. Not the most elegant approach but it would work.
A more elegant approach would be to use command system() and run your tcl script in HyperWorks from there. Execution mode can be set to synchronous using the keyword 'sync'. This command will stop the script from further execution until process ends and the file has been created.
When I automate workflows that involve other Altair products, I always try to use callaltairbatch as a first option. I can't recall right now if 'sync' can be given as a keyword there just the same, but it's worth a try.
Rafael
Ok, two answer at least part of my own question - it is not so simple to find examples for oml.
Anyway if I went for the last format, repeating the set id always on column 1. I would get there
with:
setdata=csvread('sets.csv'); tmpsetdata=setdata(setdata(:,1) == 100,:); %gets matrix part where the first column is 100 (set id = 100) tmpsetdata(:,1) = 0; %sets the first column to 0 tmpsetdata=reshape(tmpsetdata,[],1); %reshape to 1 x [] matrix finalset=tmpsetdata(tmpsetdata(:,1) > 0,:); %omit the 0 valuesso, at least it is a five liner for starters.