How to reuse a model on new data each day
Altair EmployeeI defined a report file template in Monarch Classic 13.5 for a file I receive each day – Call it the Monday File. I had to use Classic because the report format was too complex to define in Monarch Complete.
After developing the report template and preparing the Table for the Monday File in Classic, I needed to do further analysis work requiring features of Monarch Complete (e.g. group by, joins, etc.) so I chose from the Classic Table ribbon the option to Open the Table in Data Prep Studio.
In Studio, I did the grouping and joining and have the resulting answer now in Monarch Complete.
I have saved the model in Monarch Classic and the Workspace in Monarch Complete and the Project in Monarch Complete.
How do I get this model to work on the file I get on Tuesday?
Answers
-
Hello Charlie,
You could do the joins in Classic as External Lookups, and the group by as a Summary.
Best wishes,
Olly
0 -
Altair Forum User said:
Hello Charlie,
You could do the joins in Classic as External Lookups, and the group by as a Summary.
Best wishes,
Olly
In other words, you recommend doing ALL the analyses in one product or the other.
I was trying to find the min and max values for an element in a data set and didn't realize those functions were available in Summary instead of using Complete for Transform.
Thanks!
0 -
Altair Forum User said:
In other words, you recommend doing ALL the analyses in one product or the other.
I was trying to find the min and max values for an element in a data set and didn't realize those functions were available in Summary instead of using Complete for Transform.
Thanks!
Hi Charlie,
As time passes the functionality available in both Data Prep and Classic is closing together but I think it's more a case of which front end fits best for what you are working with and what you are seeking to obtain.
If you have good reliable reports or database type input with well structured data then Data Prep is a quick way to grab it and pull some results.
If you have some nasty reports (especially PDFs) that need some careful investigation and maybe some extreme processing concepts then classic has the tools for that.
I suspect that most people will have both at some point.
Once you are in Classic to deal with a problem input file then there is probably a case for staying in classic - partly because should any further development be required you only need to tweak things in one place and if you are taking the process through Server at some point a classic Project is a ready starting point.
But as I said - things are moving and the recommendation may swing a different way as future releases role out.
I should mention that I write this as a Classic user with the best part of 20 years of Monarch modelling behind me so my familiarisation and comfort zone sit naturally in the Classic space. However I can see some wider usage for self service as Data Prep provides within an Enterprise that has recognised the broad potential of the Datawatch concepts and would like to make them available to a wide range of staff on a self service basis. Even better with automation deployed of course!
Grant
0 -
Altair Forum User said:
Hi Charlie,
As time passes the functionality available in both Data Prep and Classic is closing together but I think it's more a case of which front end fits best for what you are working with and what you are seeking to obtain.
If you have good reliable reports or database type input with well structured data then Data Prep is a quick way to grab it and pull some results.
If you have some nasty reports (especially PDFs) that need some careful investigation and maybe some extreme processing concepts then classic has the tools for that.
I suspect that most people will have both at some point.
Once you are in Classic to deal with a problem input file then there is probably a case for staying in classic - partly because should any further development be required you only need to tweak things in one place and if you are taking the process through Server at some point a classic Project is a ready starting point.
But as I said - things are moving and the recommendation may swing a different way as future releases role out.
I should mention that I write this as a Classic user with the best part of 20 years of Monarch modelling behind me so my familiarisation and comfort zone sit naturally in the Classic space. However I can see some wider usage for self service as Data Prep provides within an Enterprise that has recognised the broad potential of the Datawatch concepts and would like to make them available to a wide range of staff on a self service basis. Even better with automation deployed of course!
Grant
Grant,
Thanks - You can tell I am a relatively new user and I got the impression from the existence of the option on the Classic Ribbon to Open the Table in Data Prep Studio that moving between the two was OK. We do intend to move this model into the Server at some point in the future, so your advice, and that of Olly Bond, are really helpful in setting me 'straight'.
Charlie
0 -
Charlie and Grant,
The convergence of functionality between Classic and Data Prep is real and ongoing. As of version 14, Data Prep Studio now has the ability to open Classic Models and Projects.
What this means is that the report you had to model in Classic, (using floating traps/regular expression traps/page header templates/exclusion templates/etcetera) can be included and saved as part of one single Workspace in DPS, along with any other data tables (or Models) needed, plus appends/transformations/joins, and other "data preparation" operations that may be done.
I would suggest you take a look at V14 to see exactly how the integration gap is closing between the two components of Monarch!
Good luck!
Stephen
0 -
Altair Forum User said:
Grant,
Thanks - You can tell I am a relatively new user and I got the impression from the existence of the option on the Classic Ribbon to Open the Table in Data Prep Studio that moving between the two was OK. We do intend to move this model into the Server at some point in the future, so your advice, and that of Olly Bond, are really helpful in setting me 'straight'.
Charlie
Hi Charlie,
I was going to observe that the ability to tackle the more challenging problems that one can come across with reports and especially PDF file interpretation was something that Data Prep, intended as a simple to use Date Preparation user interface, has not been intending to address. However the ability to take on those challenges in Classic and then make the results available to anyone working through Data Prep Studio adds nicely to the potential and of course fits well with the concept of socialization of data within an enterprise as offered by version 14 in particular.
However Stephen got there before me!
The key here, for my money, is that DPS can now offer an enterprise the option to use its more expert users to solve the sort of challenging data extraction problems that Classic can address and make the results of that work available to those who have no real or regular need to discover the deeper capability of Monarch but are expert at working with data that are made available to them.
So if you are creating a "production" process that will be run through Datawatch Server (Automator) "lights out" every night/week/month and then delivered to those who need the output then Classic is the traditional way to go and makes sense.
If the activity is more of a hands on investigation at a departmental level then DPS will probably be a more appealing User Interface to those undertaking the work and, as Stephen says, the ability to then work within the DPS Workspace to easily connect data from multiple sources has significant appeal.
There's no real right or wrong here - just a flexible approach that allows users and organisations to go whichever way suit them best.
V14 takes things some steps further than V13.5 as you would expect but what is already available to you is likely enough to be able experiment with both approaches an gain a appreciation for where each will fit into your future Monarch operational strategy.
HTH.
Grant
0 -
Altair Forum User said:
Grant,
Thanks - You can tell I am a relatively new user and I got the impression from the existence of the option on the Classic Ribbon to Open the Table in Data Prep Studio that moving between the two was OK. We do intend to move this model into the Server at some point in the future, so your advice, and that of Olly Bond, are really helpful in setting me 'straight'.
Charlie
Charlie,
I have just realised that we don't seem to have answered the primary part of your initial question - how do you use the model for the next day's report!
Basically, no matter whether you use DPS or Classic, you need a way to choose the data source(s) you are dealing with.
With Classic or DPS you can do this manually by open the report and apply the model in classic or the equivalent steps in DPS.
If you have saved a Project it will open both file and model for you. You could set the file to be something at a generic location and then make sure that the new day's file is always delivered there with a specific name ... or simply open the model with whatever file there is associated with it and then select the one you really want for the day.
If using Server at a later point in time you will have built in controls for the process to set that up when run.
Those are the basic concepts but there are more things one can do - probably too many to list before asking the question - "How would you like to do it?"
For example - if you were analyzing a report on a daily basis and then doing and end of period or end of month version as well on option might be to simply append the daily report to a monthly or period based file every day of the period and set a filter in the model that by default sets itself to "Today". If necessary re-running an earlier day would be simple to set up.
A monthly analysis would just skip the filter completely.
Archive the file at the end of the month if required, save the project associated with a new file for the next period and continue form there.
Server/Automator can take all of that to a new level of course so if you are heading in that direction it would make sense to do something today that, even if run manually, is the basis for later automation and goes a long way toward a proof of process thus making automation an easier task when the time arrives.
I would like to be more specific on this but there are a number of options and paths available and I don't think it would be true benefit to imply that there is only one approach you could consider - though the answer to the question above may lead us to such a conclusion!
HTH.
Grant
0 -
Altair Forum User said:
Charlie and Grant,
The convergence of functionality between Classic and Data Prep is real and ongoing. As of version 14, Data Prep Studio now has the ability to open Classic Models and Projects.
What this means is that the report you had to model in Classic, (using floating traps/regular expression traps/page header templates/exclusion templates/etcetera) can be included and saved as part of one single Workspace in DPS, along with any other data tables (or Models) needed, plus appends/transformations/joins, and other "data preparation" operations that may be done.
I would suggest you take a look at V14 to see exactly how the integration gap is closing between the two components of Monarch!
Good luck!
Stephen
Thanks for the update on 14 - I didn't notice that in the literature. -Charlie
0 -
Altair Forum User said:
Charlie,
I have just realised that we don't seem to have answered the primary part of your initial question - how do you use the model for the next day's report!
Basically, no matter whether you use DPS or Classic, you need a way to choose the data source(s) you are dealing with.
With Classic or DPS you can do this manually by open the report and apply the model in classic or the equivalent steps in DPS.
If you have saved a Project it will open both file and model for you. You could set the file to be something at a generic location and then make sure that the new day's file is always delivered there with a specific name ... or simply open the model with whatever file there is associated with it and then select the one you really want for the day.
If using Server at a later point in time you will have built in controls for the process to set that up when run.
Those are the basic concepts but there are more things one can do - probably too many to list before asking the question - "How would you like to do it?"
For example - if you were analyzing a report on a daily basis and then doing and end of period or end of month version as well on option might be to simply append the daily report to a monthly or period based file every day of the period and set a filter in the model that by default sets itself to "Today". If necessary re-running an earlier day would be simple to set up.
A monthly analysis would just skip the filter completely.
Archive the file at the end of the month if required, save the project associated with a new file for the next period and continue form there.
Server/Automator can take all of that to a new level of course so if you are heading in that direction it would make sense to do something today that, even if run manually, is the basis for later automation and goes a long way toward a proof of process thus making automation an easier task when the time arrives.
I would like to be more specific on this but there are a number of options and paths available and I don't think it would be true benefit to imply that there is only one approach you could consider - though the answer to the question above may lead us to such a conclusion!
HTH.
Grant
Thanks -Charlie
0 -
Altair Forum User said:
Charlie and Grant,
The convergence of functionality between Classic and Data Prep is real and ongoing. As of version 14, Data Prep Studio now has the ability to open Classic Models and Projects.
What this means is that the report you had to model in Classic, (using floating traps/regular expression traps/page header templates/exclusion templates/etcetera) can be included and saved as part of one single Workspace in DPS, along with any other data tables (or Models) needed, plus appends/transformations/joins, and other "data preparation" operations that may be done.
I would suggest you take a look at V14 to see exactly how the integration gap is closing between the two components of Monarch!
Good luck!
Stephen
Stephen
Thanks
I have downloaded V14 and it is as you say - more converged than in 13.5. I was able to bring in a model developed in Classic 13.5 into Monarch Complete 14 and use it on a new data file. Nice.
Looking forward now to getting Server Automation 14 working here! -Charlie
0