Automation Runs Very Slow when Reading Large Data AS400 Data Sets via ODBC
I have created automation using Auto Com (Command Line automation) where Monarch 15.x is reading large data sets from the AS400 via ODBC connection. The file sizes range from a few hundred records to over 3 million (15 to 30 fields). As the number of records get above 50,000 the automation began to crawl. The automation always complete successfully, but the wait time is painfully long. The automation is set up to run during off hours, which is ok, but these automation scripts are also set up to run manually, on demand.
Note that I am using Monarch Classic to the the ODBC linking and processing. My question is, firstly, could I integrate Data Prep Studio into the automation (again, we only have the Auto Com license). If DPS could read the AS400 tables much faster, and create the required .csv file exports, hopefully that would speed up the automation.
Any advice would be helpful.
------------------------------
Al Rice
------------------------------
Note that I am using Monarch Classic to the the ODBC linking and processing. My question is, firstly, could I integrate Data Prep Studio into the automation (again, we only have the Auto Com license). If DPS could read the AS400 tables much faster, and create the required .csv file exports, hopefully that would speed up the automation.
Any advice would be helpful.
------------------------------
Al Rice
------------------------------