{ "date": "2016-01-18T21:39:14", "slug": "datapump-processing-object-type-misleading-messages", "link": "https://www.dbi-services.com/blog/datapump-processing-object-type-misleading-messages/", "title": { "rendered": "DataPump ‘Processing object type’ misleading messages" }, "content": { "rendered": "

By Franck Pachot

\n

.
\nYou’ve started a long DataPump and see it stuck on TABLE/STATISTICS/TABLE_STATISTICS, but you don’t expect that step to take a long time. Let’s see if we can rely on that message.
\n
\nI’ve a 500MB table and export it. I’ll use the 12c LOGTIME option to print a timestamp in front of each output line:

\n
\n18-JAN-16 21:35:44.038: Starting \"SOE\".\"SYS_EXPORT_TABLE_01\":  soe/********@//localhost/SWINGBENCH tables=soe.DEMO directory=tmp status=1 logtime=all reuse_dumpfiles=y\n18-JAN-16 21:35:48.294: Processing object type TABLE_EXPORT/TABLE/TABLE\n18-JAN-16 21:35:48.687: Processing object type TABLE_EXPORT/TABLE/TABLE_DATA\n18-JAN-16 21:35:49.532: Processing object type TABLE_EXPORT/TABLE/INDEX/INDEX\n18-JAN-16 21:35:49.849: Processing object type TABLE_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS\n18-JAN-16 21:35:50.242: Processing object type TABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS\n18-JAN-16 21:35:54.779: . . exported \"SOE\".\"DEMO\"                                542.6 MB 5051504 rows\n18-JAN-16 21:35:55.257: Processing object type TABLE_EXPORT/TABLE/STATISTICS/MARKER\n18-JAN-16 21:35:55.644: Master table \"SOE\".\"SYS_EXPORT_TABLE_01\" successfully loaded/unloaded\n18-JAN-16 21:35:55.693: ******************************************************************************\n18-JAN-16 21:35:55.694: Dump file set for SOE.SYS_EXPORT_TABLE_01 is:\n18-JAN-16 21:35:55.697:   /tmp/expdat.dmp\n18-JAN-16 21:35:55.738: Job \"SOE\".\"SYS_EXPORT_TABLE_01\" successfully completed at Mon Jan 18 21:35:55 2016 elapsed 0 00:00:13\n
\n

If you rely on the ‘Processing’ messages, the export of table data takes 49.532 – 48.687 = 0.845 seconds. And exporting table statistics takes 55.257 – 50.242 = 5 seconds. Obviously, this is wrong.

\n

When you look at the ‘exported … rows’ it seems that the export of table data was still running at that time.

\n

As you can see above, I’ve run the expdp with the STATUS=1 in order to display job status every 1 second.

\n

After the TABLE_EXPORT/TABLE/TABLE_DATA the worker was not executing:

\n
\nWorker 1 Status:\n  Instance ID: 1\n  Instance name: CDB\n  Host name: VM117\n  Object start time: Monday, 18 January, 2016 21:35:48\n  Object status at: Monday, 18 January, 2016 21:35:48\n  Process Name: DW00\n  State: WORK WAITING\n
\n

Then it changed to the state of EXECUTING for several seconds.

\n

Only at 21:35:51 we can see a status showing that it’s working on the table export, which is more than 1 second after the related ‘Processing’ message:

\n
\nWorker 1 Status:\n  Instance ID: 1\n  Instance name: CDB\n  Host name: VM117\n  Object start time: Monday, 18 January, 2016 21:35:51\n  Object status at: Monday, 18 January, 2016 21:35:51\n  Process Name: DW00\n  State: EXECUTING\n  Object Schema: SOE\n  Object Name: DEMO\n  Object Type: TABLE_EXPORT/TABLE/TABLE_DATA\n  Completed Objects: 1\n  Total Objects: 1\n  Completed Rows: 1,398,529\n  Worker Parallelism: 1\n
\n

Then, the status displayed every seconds shows the same with an increasing number of rows until 21:35:54:

\n
\nWorker 1 Status:\n  Instance ID: 1\n  Instance name: CDB\n  Host name: VM117\n  Access method: direct_path\n  Object start time: Monday, 18 January, 2016 21:35:51\n  Object status at: Monday, 18 January, 2016 21:35:54\n  Process Name: DW00\n  State: EXECUTING\n  Object Schema: SOE\n  Object Name: DEMO\n  Object Type: TABLE_EXPORT/TABLE/TABLE_DATA\n  Completed Objects: 1\n  Total Objects: 1\n  Completed Rows: 5,051,504\n  Completed Bytes: 569,032,224\n  Percent Done: 100\n  Worker Parallelism: 1\n
\n

and this is where we get the message:

\n
\n18-JAN-16 21:35:54.779: . . exported \"SOE\".\"DEMO\"                                542.6 MB 5051504 rows\n
\n

Then we can see the following status which suggests that TABLE_EXPORT/TABLE/STATISTICS/MARKER has started:

\n
\nWorker 1 Status:\n  Instance ID: 1\n  Instance name: CDB\n  Host name: VM117\n  Object start time: Monday, 18 January, 2016 21:35:51\n  Object status at: Monday, 18 January, 2016 21:35:55\n  Process Name: DW00\n  State: EXECUTING\n  Object Schema: SYS\n  Object Type: TABLE_EXPORT/TABLE/STATISTICS/MARKER\n  Worker Parallelism: 1\n
\n

but we see the related ‘Processing’ message only after it:

\n
\n18-JAN-16 21:35:55.257: Processing object type TABLE_EXPORT/TABLE/STATISTICS/MARKER\n
\n

Strange to take 4 seconds on that step, and anyway the ‘Processing…’ message comes at the end here. Actually, this step is waiting for the previous one to finish.

\n

My first conclusion here is that you should not rely on the ‘Processing…’ messages to know what is currently running.
\nIf you’ve run expdp from your terminal, you can interrupt it with control-C and you have a CLI to control the worker processes. They are still running and you can see the status with STATUS and then continue to the previous mode with CONTINUE_CLIENT.
\nIf expdp is running in background, you can attach to the job and do the same.
\nRemember that DataPump is running through background jobs, which explains that the ‘Processing’ message may not be in sync with what is currently processed.

\n

So, the ‘LOGTIME’ option a bit useless when it puts the timestamp in front the ‘Processing’ messages. However, it’s useful for the ‘exported’ message as it is the end of the table export.

\n", "protected": false } }