Need suggestions on structuring this data
Hey guys,
I need input on how to store and analyze the data I am collecting. Basically, I am monitoring about 900 different "bins." Every ten minutes I record the number of items in each different bin, and the "alpha level" associated with each bin. Each bin has a maximum number of objects, and I've casually noticed that some bins hit max units and never fluctuate, while others flow up and down.
Right now I have it structured as follows (saved as a CSV file that appends every run):
Bin Name, Alpha Level, Bin Item Count, Bin Item Capacity, Date of recording
Again, my goals are :
1. Find Trends in alpha levels and item counts
2. Efficiently store and analyze my data ( I have limited resources - both computing power and bandwidth)
Suggestions? TYIA
EDIT: Alpha Level, Bin Item Count, and Bin Capacity are real numbers, Bin Name is a string, and Date is obviously a date.
I need input on how to store and analyze the data I am collecting. Basically, I am monitoring about 900 different "bins." Every ten minutes I record the number of items in each different bin, and the "alpha level" associated with each bin. Each bin has a maximum number of objects, and I've casually noticed that some bins hit max units and never fluctuate, while others flow up and down.
Right now I have it structured as follows (saved as a CSV file that appends every run):
Bin Name, Alpha Level, Bin Item Count, Bin Item Capacity, Date of recording
Again, my goals are :
1. Find Trends in alpha levels and item counts
2. Efficiently store and analyze my data ( I have limited resources - both computing power and bandwidth)
Suggestions? TYIA
EDIT: Alpha Level, Bin Item Count, and Bin Capacity are real numbers, Bin Name is a string, and Date is obviously a date.
Find more posts tagged with
Sort by:
1 - 2 of
21
I am aware of it Marius, but I'm having trouble finding the necessary documentation to build the process. If you have a recommended tutorial or packet of material it would be helpful.
Marius wrote:
Hi Jester,
are you aware of our Value Series Extension? It is made for analyzing series of data, as you actually have: for each bin, the item numbers form a series of values over time. If you need help with the series extension, please post your issues here.
Best regards,
Marius
Also, I need to know how I should store my mined data. It's not overly complex, but I need to make sure I am storing it optimally because I don't want to have to reformat it later when the amount collected is in the GB size. So, do you have any resources talking about how best to store the information mined?
are you aware of our Value Series Extension? It is made for analyzing series of data, as you actually have: for each bin, the item numbers form a series of values over time. If you need help with the series extension, please post your issues here.
Best regards,
Marius