How to trap data from thousands of report files?
Altair Employee------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
Answers
-
Hi Akitake,
If a single model or work space is suitable for all reports, then Monarch can process around 1000 reports in a single session. Of course, it may take a long time to load all of these - especially PDF files. For a few thousand, you should be able to run several batches and use the Append function when exporting to create a single output file.
For many thousands of files, or if this is an ongoing requirement, I would suggest using the server version (Automator). You build the model or workspace on a sample, then deploy it onto the server version. This will process as many files as you need and is only limited by the resources of the server.
Regards,
Steve.
------------------------------
Steve Caiels
Professional Services
Altair
------------------------------
-------------------------------------------
Original Message:
Sent: 11-13-2019 12:41 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
I have thousands of PDF report files and want to trap data from them and combine into one. I know it's easy to trap data from one or a few report files with Monarch, but how can I get scalability to handreds or thousands of files?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
"0 -
Just to add on to Steve's answer, if you are using Monarch Classic (and have the appropriate license) you can use COM automation where you can call monarch from a command line. This can be used in a batch file where you can process each file individually and append the output to a common export.Steve Caiels_21881 said:Hi Akitake,
If a single model or work space is suitable for all reports, then Monarch can process around 1000 reports in a single session. Of course, it may take a long time to load all of these - especially PDF files. For a few thousand, you should be able to run several batches and use the Append function when exporting to create a single output file.
For many thousands of files, or if this is an ongoing requirement, I would suggest using the server version (Automator). You build the model or workspace on a sample, then deploy it onto the server version. This will process as many files as you need and is only limited by the resources of the server.
Regards,
Steve.
------------------------------
Steve Caiels
Professional Services
Altair
------------------------------
-------------------------------------------
Original Message:
Sent: 11-13-2019 12:41 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
I have thousands of PDF report files and want to trap data from them and combine into one. I know it's easy to trap data from one or a few report files with Monarch, but how can I get scalability to handreds or thousands of files?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
"
------------------------------
Chris Porthouse
Senior Implementation & Integration Engineer
Altair
------------------------------
-------------------------------------------
Original Message:
Sent: 11-14-2019 03:45 AM
From: Steve Caiels
Subject: How to trap data from thousands of report files?
Hi Akitake,
If a single model or work space is suitable for all reports, then Monarch can process around 1000 reports in a single session. Of course, it may take a long time to load all of these - especially PDF files. For a few thousand, you should be able to run several batches and use the Append function when exporting to create a single output file.
For many thousands of files, or if this is an ongoing requirement, I would suggest using the server version (Automator). You build the model or workspace on a sample, then deploy it onto the server version. This will process as many files as you need and is only limited by the resources of the server.
Regards,
Steve.
------------------------------
Steve Caiels
Professional Services
Altair
------------------------------
Original Message:
Sent: 11-13-2019 12:41 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
I have thousands of PDF report files and want to trap data from them and combine into one. I know it's easy to trap data from one or a few report files with Monarch, but how can I get scalability to handreds or thousands of files?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
"0 -
Hi Steve,Steve Caiels_21881 said:Hi Akitake,
If a single model or work space is suitable for all reports, then Monarch can process around 1000 reports in a single session. Of course, it may take a long time to load all of these - especially PDF files. For a few thousand, you should be able to run several batches and use the Append function when exporting to create a single output file.
For many thousands of files, or if this is an ongoing requirement, I would suggest using the server version (Automator). You build the model or workspace on a sample, then deploy it onto the server version. This will process as many files as you need and is only limited by the resources of the server.
Regards,
Steve.
------------------------------
Steve Caiels
Professional Services
Altair
------------------------------
-------------------------------------------
Original Message:
Sent: 11-13-2019 12:41 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
I have thousands of PDF report files and want to trap data from them and combine into one. I know it's easy to trap data from one or a few report files with Monarch, but how can I get scalability to handreds or thousands of files?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
"
Thank you for your reply. Yes, a single model is fine to me, but how can I easily add thousands of report files to a table? I know by selecting the menu 'Report Options' > 'Add Report' of the table, I can add a report to the table. But I'm afraid I can't add multiple files at once. Is there a good alternative?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
-------------------------------------------
Original Message:
Sent: 11-14-2019 03:45 AM
From: Steve Caiels
Subject: How to trap data from thousands of report files?
Hi Akitake,
If a single model or work space is suitable for all reports, then Monarch can process around 1000 reports in a single session. Of course, it may take a long time to load all of these - especially PDF files. For a few thousand, you should be able to run several batches and use the Append function when exporting to create a single output file.
For many thousands of files, or if this is an ongoing requirement, I would suggest using the server version (Automator). You build the model or workspace on a sample, then deploy it onto the server version. This will process as many files as you need and is only limited by the resources of the server.
Regards,
Steve.
------------------------------
Steve Caiels
Professional Services
Altair
------------------------------
Original Message:
Sent: 11-13-2019 12:41 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
I have thousands of PDF report files and want to trap data from them and combine into one. I know it's easy to trap data from one or a few report files with Monarch, but how can I get scalability to handreds or thousands of files?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
"0 -
Hi Chris,Chris Porthouse said:Just to add on to Steve's answer, if you are using Monarch Classic (and have the appropriate license) you can use COM automation where you can call monarch from a command line. This can be used in a batch file where you can process each file individually and append the output to a common export.
------------------------------
Chris Porthouse
Senior Implementation & Integration Engineer
Altair
------------------------------
-------------------------------------------
Original Message:
Sent: 11-14-2019 03:45 AM
From: Steve Caiels
Subject: How to trap data from thousands of report files?
Hi Akitake,
If a single model or work space is suitable for all reports, then Monarch can process around 1000 reports in a single session. Of course, it may take a long time to load all of these - especially PDF files. For a few thousand, you should be able to run several batches and use the Append function when exporting to create a single output file.
For many thousands of files, or if this is an ongoing requirement, I would suggest using the server version (Automator). You build the model or workspace on a sample, then deploy it onto the server version. This will process as many files as you need and is only limited by the resources of the server.
Regards,
Steve.
------------------------------
Steve Caiels
Professional Services
Altair
------------------------------
Original Message:
Sent: 11-13-2019 12:41 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
I have thousands of PDF report files and want to trap data from them and combine into one. I know it's easy to trap data from one or a few report files with Monarch, but how can I get scalability to handreds or thousands of files?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
"
Thank you for your advise. Actually I have not only PDF reports but also some Excel files, which I want to combine. However, I think COM automation is worth trying. Couldn't find any documents in my latest Monarch 15 installation but found in Monarch 14. I will look into it.
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
Japan
81362255826
------------------------------
-------------------------------------------
Original Message:
Sent: 11-14-2019 08:59 AM
From: Chris Porthouse
Subject: How to trap data from thousands of report files?
Just to add on to Steve's answer, if you are using Monarch Classic (and have the appropriate license) you can use COM automation where you can call monarch from a command line. This can be used in a batch file where you can process each file individually and append the output to a common export.
------------------------------
Chris Porthouse
Senior Implementation & Integration Engineer
Altair
------------------------------
Original Message:
Sent: 11-14-2019 03:45 AM
From: Steve Caiels
Subject: How to trap data from thousands of report files?
Hi Akitake,
If a single model or work space is suitable for all reports, then Monarch can process around 1000 reports in a single session. Of course, it may take a long time to load all of these - especially PDF files. For a few thousand, you should be able to run several batches and use the Append function when exporting to create a single output file.
For many thousands of files, or if this is an ongoing requirement, I would suggest using the server version (Automator). You build the model or workspace on a sample, then deploy it onto the server version. This will process as many files as you need and is only limited by the resources of the server.
Regards,
Steve.
------------------------------
Steve Caiels
Professional Services
Altair
Original Message:
Sent: 11-13-2019 12:41 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
I have thousands of PDF report files and want to trap data from them and combine into one. I know it's easy to trap data from one or a few report files with Monarch, but how can I get scalability to handreds or thousands of files?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
"0 -
Hi Akitake,Altair Forum User said:Hi Steve,
Thank you for your reply. Yes, a single model is fine to me, but how can I easily add thousands of report files to a table? I know by selecting the menu 'Report Options' > 'Add Report' of the table, I can add a report to the table. But I'm afraid I can't add multiple files at once. Is there a good alternative?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
-------------------------------------------
Original Message:
Sent: 11-14-2019 03:45 AM
From: Steve Caiels
Subject: How to trap data from thousands of report files?
Hi Akitake,
If a single model or work space is suitable for all reports, then Monarch can process around 1000 reports in a single session. Of course, it may take a long time to load all of these - especially PDF files. For a few thousand, you should be able to run several batches and use the Append function when exporting to create a single output file.
For many thousands of files, or if this is an ongoing requirement, I would suggest using the server version (Automator). You build the model or workspace on a sample, then deploy it onto the server version. This will process as many files as you need and is only limited by the resources of the server.
Regards,
Steve.
------------------------------
Steve Caiels
Professional Services
Altair
------------------------------
Original Message:
Sent: 11-13-2019 12:41 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
I have thousands of PDF report files and want to trap data from them and combine into one. I know it's easy to trap data from one or a few report files with Monarch, but how can I get scalability to handreds or thousands of files?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
"
Drag and Drop should do it. Just grab the files and make sure you drop them into the green highlight that will show up when you get the mouse over the original file.
Regards,
Steve,
------------------------------
Steve Caiels
Professional Services
Altair
------------------------------
-------------------------------------------
Original Message:
Sent: 11-20-2019 12:47 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
Hi Steve,
Thank you for your reply. Yes, a single model is fine to me, but how can I easily add thousands of report files to a table? I know by selecting the menu 'Report Options' > 'Add Report' of the table, I can add a report to the table. But I'm afraid I can't add multiple files at once. Is there a good alternative?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
Original Message:
Sent: 11-14-2019 03:45 AM
From: Steve Caiels
Subject: How to trap data from thousands of report files?
Hi Akitake,
If a single model or work space is suitable for all reports, then Monarch can process around 1000 reports in a single session. Of course, it may take a long time to load all of these - especially PDF files. For a few thousand, you should be able to run several batches and use the Append function when exporting to create a single output file.
For many thousands of files, or if this is an ongoing requirement, I would suggest using the server version (Automator). You build the model or workspace on a sample, then deploy it onto the server version. This will process as many files as you need and is only limited by the resources of the server.
Regards,
Steve.
------------------------------
Steve Caiels
Professional Services
Altair
Original Message:
Sent: 11-13-2019 12:41 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
I have thousands of PDF report files and want to trap data from them and combine into one. I know it's easy to trap data from one or a few report files with Monarch, but how can I get scalability to handreds or thousands of files?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
"0 -
Thank you for your advice Steve. It works perfectly in my case. I love Monarch!Steve Caiels_21881 said:Hi Akitake,
Drag and Drop should do it. Just grab the files and make sure you drop them into the green highlight that will show up when you get the mouse over the original file.
Regards,
Steve,
------------------------------
Steve Caiels
Professional Services
Altair
------------------------------
-------------------------------------------
Original Message:
Sent: 11-20-2019 12:47 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
Hi Steve,
Thank you for your reply. Yes, a single model is fine to me, but how can I easily add thousands of report files to a table? I know by selecting the menu 'Report Options' > 'Add Report' of the table, I can add a report to the table. But I'm afraid I can't add multiple files at once. Is there a good alternative?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
Original Message:
Sent: 11-14-2019 03:45 AM
From: Steve Caiels
Subject: How to trap data from thousands of report files?
Hi Akitake,
If a single model or work space is suitable for all reports, then Monarch can process around 1000 reports in a single session. Of course, it may take a long time to load all of these - especially PDF files. For a few thousand, you should be able to run several batches and use the Append function when exporting to create a single output file.
For many thousands of files, or if this is an ongoing requirement, I would suggest using the server version (Automator). You build the model or workspace on a sample, then deploy it onto the server version. This will process as many files as you need and is only limited by the resources of the server.
Regards,
Steve.
------------------------------
Steve Caiels
Professional Services
Altair
Original Message:
Sent: 11-13-2019 12:41 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
I have thousands of PDF report files and want to trap data from them and combine into one. I know it's easy to trap data from one or a few report files with Monarch, but how can I get scalability to handreds or thousands of files?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
"
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
Japan
81362255826
------------------------------
-------------------------------------------
Original Message:
Sent: 11-21-2019 03:08 AM
From: Steve Caiels
Subject: How to trap data from thousands of report files?
Hi Akitake,
Drag and Drop should do it. Just grab the files and make sure you drop them into the green highlight that will show up when you get the mouse over the original file.
Regards,
Steve,
------------------------------
Steve Caiels
Professional Services
Altair
------------------------------
Original Message:
Sent: 11-20-2019 12:47 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
Hi Steve,
Thank you for your reply. Yes, a single model is fine to me, but how can I easily add thousands of report files to a table? I know by selecting the menu 'Report Options' > 'Add Report' of the table, I can add a report to the table. But I'm afraid I can't add multiple files at once. Is there a good alternative?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
Original Message:
Sent: 11-14-2019 03:45 AM
From: Steve Caiels
Subject: How to trap data from thousands of report files?
Hi Akitake,
If a single model or work space is suitable for all reports, then Monarch can process around 1000 reports in a single session. Of course, it may take a long time to load all of these - especially PDF files. For a few thousand, you should be able to run several batches and use the Append function when exporting to create a single output file.
For many thousands of files, or if this is an ongoing requirement, I would suggest using the server version (Automator). You build the model or workspace on a sample, then deploy it onto the server version. This will process as many files as you need and is only limited by the resources of the server.
Regards,
Steve.
------------------------------
Steve Caiels
Professional Services
Altair
Original Message:
Sent: 11-13-2019 12:41 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
I have thousands of PDF report files and want to trap data from them and combine into one. I know it's easy to trap data from one or a few report files with Monarch, but how can I get scalability to handreds or thousands of files?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
"0 -
Hi Akitake,Altair Forum User said:Hi Steve,
Thank you for your reply. Yes, a single model is fine to me, but how can I easily add thousands of report files to a table? I know by selecting the menu 'Report Options' > 'Add Report' of the table, I can add a report to the table. But I'm afraid I can't add multiple files at once. Is there a good alternative?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
-------------------------------------------
Original Message:
Sent: 11-14-2019 03:45 AM
From: Steve Caiels
Subject: How to trap data from thousands of report files?
Hi Akitake,
If a single model or work space is suitable for all reports, then Monarch can process around 1000 reports in a single session. Of course, it may take a long time to load all of these - especially PDF files. For a few thousand, you should be able to run several batches and use the Append function when exporting to create a single output file.
For many thousands of files, or if this is an ongoing requirement, I would suggest using the server version (Automator). You build the model or workspace on a sample, then deploy it onto the server version. This will process as many files as you need and is only limited by the resources of the server.
Regards,
Steve.
------------------------------
Steve Caiels
Professional Services
Altair
------------------------------
Original Message:
Sent: 11-13-2019 12:41 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
I have thousands of PDF report files and want to trap data from them and combine into one. I know it's easy to trap data from one or a few report files with Monarch, but how can I get scalability to handreds or thousands of files?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
"
Copy all PDF files in one folder. Launch DPS, select your model, in the prepare page under the table name (which should be the name of the model) select report option, select Add Report, navigate to the folder where all PDF files resides, select the first file, scroll to the bottom, hold the shift key, select the last file. This should select all files, then click on the Open button. The model will be applied to all PDF files that are opened in DPS and the table will display all extracted data from all PDF document.
The limit for open reports in one Monarch session for classic is 1024. You can check the Monarch specifications by clicking the following links:
Monarch Classic specifications: http://docs.datawatch.com/monarch/desktop/ht_cui_specs.htm
Data Prep Specifications: http://docs.datawatch.com/monarch/desktop/DPS_Technical_Specifications.htm
Regards
Mo
------------------------------
Mahmoud Abdolrahim
Senior Implementation & Integration Engineer
Datawatch Corporation
MA
(978) 935-3840
------------------------------
-------------------------------------------
Original Message:
Sent: 11-20-2019 12:47 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
Hi Steve,
Thank you for your reply. Yes, a single model is fine to me, but how can I easily add thousands of report files to a table? I know by selecting the menu 'Report Options' > 'Add Report' of the table, I can add a report to the table. But I'm afraid I can't add multiple files at once. Is there a good alternative?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
Original Message:
Sent: 11-14-2019 03:45 AM
From: Steve Caiels
Subject: How to trap data from thousands of report files?
Hi Akitake,
If a single model or work space is suitable for all reports, then Monarch can process around 1000 reports in a single session. Of course, it may take a long time to load all of these - especially PDF files. For a few thousand, you should be able to run several batches and use the Append function when exporting to create a single output file.
For many thousands of files, or if this is an ongoing requirement, I would suggest using the server version (Automator). You build the model or workspace on a sample, then deploy it onto the server version. This will process as many files as you need and is only limited by the resources of the server.
Regards,
Steve.
------------------------------
Steve Caiels
Professional Services
Altair
Original Message:
Sent: 11-13-2019 12:41 AM
From: Akitake Oguchi
Subject: How to trap data from thousands of report files?
I have thousands of PDF report files and want to trap data from them and combine into one. I know it's easy to trap data from one or a few report files with Monarch, but how can I get scalability to handreds or thousands of files?
------------------------------
Akitake Oguchi
Technical Manager
Altair Engineering Inc.
CHUO-KU
81362255826
------------------------------
"0