EXTRACT JOURNAL

The EXTRACT JOURNAL utility conceptually extracts the most recent AFTR image for each dbkey recorded on an archived journal file and writes it to an extract file. The extract file can later be used as input to a ROLLFORWARD command for a "quick" recovery of a database area or file.
idmscu19
The EXTRACT JOURNAL utility conceptually extracts the most recent AFTR image for each dbkey recorded on an archived journal file and writes it to an extract file. The extract file can later be used as input to a ROLLFORWARD command for a "quick" recovery of a database area or file.
When used as part of a regular procedure to extract journal images from archived journal tapes, it allows for a quicker recovery during a critical time frame by doing much of the work during a non-critical time frame. It also speeds up recovery by extracting only the journal images it needs to recover the areas or files selected.
This article describes the following information:
2
2
Authorization
DBAWRITE authority is needed for all options.
Syntax
  ►►─── EXTRACT JOURNAL FOR ─┬─ DMCL ──────────────────────────────┬────────────►                            │        ┌──────────── , ───────────┐ │                            ├─ FILE ─▼─ segment-name.file-name ─┴─┤                            │        ┌──────────── , ───────────┐ │                            ├─ AREA ─▼─ segment-name.area-name ─┴─┤                            │           ┌─────── , ──────┐        │                            └─ SEGMENT ─▼─ segment-name ─┴────────┘  ►─┬─ COMPLETED ─┬──┬────────────────────┬────────────────────────────────────►    └─ ALL ───────┘  └─ TO ─┬─ ddname ────┤                            ├─ file-name ─┤                            ├─ link-name ─┤                            └─ SYS002 ◄───┘    ►─┬──────────────────────────────────────┬───────────────────────────────────►    └─ VERIFY QUIESCE ──┬───────────┬──────┘                        ├─ NONE ────┤                        ├─ LIMITED ─┤                        └─ FULL ────┘  ►─┬─────────┬────────────────────────────────────────────────────────────────►    └─ UTC────┘    ►──┬─────────────────────────────────────────────────┬───────────────────────►     └─ START AT ── ' ─┬─ date-time ─┬─ ' ─────────────┘                       ├─ date ──────┤                       └─ time ──────┘  ►──┬──────────────────────────────────────────────────┬──────────────────────►◄     └─ STOP AT ── ' ─┬─ date-time ─┬─ ' ───────────────┘                      ├─ date ──────┤                      └─ time ──────┘  
Parameters
  • DMCL
    Specifies that dbkeys for all areas defined in the DMCL specified in the SYSIDMS parameter file will be included in the extract file.
  • FILE
    Specifies to include
    segment-name.file-name
    dbkeys in the specified file in the extract file.
  • AREA
    Specifies to include
    segment-name.area-name
    dbkeys in the specified area in the extract file.
  • SEGMENT
    Specifies to include dbkeys for all areas defined in the specified segments in the extract file.
  • COMPLETED
    If you do not specify a STOP time, only after images for transactions that have completed by the end of the archive file will be written to the extract file. After images are written to the extract file for incomplete distributed transactions whose state is InDoubt at the end of the journal file (unless a section recovery control file entry overrides this behavior by specifying BACKOUT for one or more of the transactions).
    If you do specify a STOP time, only after images for transactions that have completed by the first checkpoint at or following the stop time will be written to the extract file. After images are written to the extract file for incomplete distributed transactions whose state is InDoubt at the stop time (unless a section recovery control file entry overrides this behavior by specifying BACKOUT for one or more of the transactions).
  • ALL
    If you do not specify a STOP time, after images for all transactions will be written to the extract file.
    If you do specify a STOP time, after images for all transactions will be written to the extract file up to the point at or after the stop time where there are no active transactions. If such a point is not found by the end of the archive file, all after images will be written.
  • TO
    Specifies the external name to be used for the extract file.
  • ddname
    The external z/OS, or CMS name for the extract file.
  • file-name
    The external z/VSE name for the extract file.
  • SYS002
    The default name for all systems.
  • VERIFY QUIESCE
    Specifies the level of quiesce point verification that will be performed.
  • NONE
    Specifies that no quiesce point verification should be done. This is the default if neither VERIFY QUIESCE nor START AT is specified. If START AT is specified, NONE is treated as LIMITED.
  • LIMITED
    Specifies that limited quiesce point verification be performed. This is the default if VERIFY QUIESCE is not specified and START AT is specified.
  • FULL
    Specifies that the strictest form of quiesce point verification be performed. This is the default if VERIFY QUIESCE is specified without indicating the verification level.
  • START AT
    Specifies that the operation should start only after reaching a specified date and time in the journal file. Only images for transactions that begin after the specified start time will be processed. By default, processing starts at the beginning of the journal file.
  • STOP AT
    Specifies that the operation should stop as soon as possible after reaching a specified date and time in the journal file.
    By default, processing stops when the end of the journal file is reached.
  • date
    Specifies the date, in one of the following formats:
    • yyyy-mm-dd
    • mm/dd/yyyy
    • dd.mm.yyyy
    In these formats, the following rules apply:
    • yyyy
      specifies the year.
      yyyy
      must be an integer in the range 0001 through 9999. Leading zeros are optional.
    • mm
      specifies the month within the year.
      mm
      must be an integer in the range 01 through 12. Leading zeros are optional.
    • dd
      specifies the day within the month.
      dd
      must be an integer in the range 01 through 31. Leading zeros are optional.
    The combined values of
    yyyy
    ,
    mm
    , and
    dd
    must represent a valid date. For example, 1988-02-29 is a valid date. 1989-02-29 is not.
  • date-time
    Specifies the date and time, where:
    • The format for specifying the DATE-TIME is:
      yyyy-mm-dd-hh.mm.ss.ffffff
    • The rules for specifying the DATE component of DATE-TIME are the same as for DATE. The rules for specifying the TIME component of DATE-TIME are:
      • hh
        specifies the hour on a 24-hour clock.
        hh
        must be an integer in the range 00 through 23. Leading zeros are optional.
      • mm
        specifies the number of minutes past the hour.
        mm
        must be an integer in the range 00 through 59. Leading zeros are optional.
      • ss
        specifies the number of seconds past the minute.
        ss
        must be an integer in the range 00 through 59. Leading zeros are optional.
      • ffffff
        specifies the number of millionths of a second past the specified second.
        ffffff
        is optional; if you include it, it must be an integer in the range 000000 through 999999. The default value is 000000. Trailing zeros are optional.
  • time
    Specifies the time in one of the following formats:
    • hh.mm.ss
    • hh:mm:ss
    The rules for specifying TIME are the same as those listed for DATE-TIME.
    When TIME is specified, the date defaults to the current date.
  • UTC
    Specifies that Start and Stop times are interpreted as UTC times instead of local times.
Usage
How to submit EXTRACT JOURNAL
EXTRACT JOURNAL must be submitted through the batch command facility and in local mode.
When to use EXTRACT JOURNAL
EXTRACT JOURNAL is most commonly used on a daily basis or after each archive journal is created. It is used as part of a plan for recovering a file or area on a device that takes a hardware hit.
For more information, see the 'FROM EXTRACT' option on the ROLLFORWARD command. It allows ROLLFORWARD processing to be done during a recovery situation when time is critical or extract processing to be done during a non-critical time in case it is needed.
How EXTRACT JOURNAL works
EXTRACT JOURNAL works much the same as ROLLFORWARD using the sort option, except that instead of applying the latest after image directly to the database, the image is written to the extract file. The input archive journal is read and after images for selected areas are sorted. Only the latest image from a completed transaction is saved, and the rest are discarded. By saving only the latest after images for only selected areas, the resulting extract file is smaller and presorted, making subsequent ROLLFORWARD processing go much faster.
If a transaction does not end on the current set of input files, all AFTR images from that transaction are written to the extract file. When these extra images are processed by a subsequent EXTRACT JOURNAL or by a ROLLFORWARD command, they are included with completed AFTR images or discarded depending on the final status of that transaction. Only the most recent AFTR image taken from a completed transaction for a given dbkey is applied to the database.
Multiple input archive journals
Multiple input archive journals can be used as input provided they are read in the correct order. They do not need to be merged into a single file. With restrictions, each archived journal can also be processed with separate EXTRACT JOURNAL jobs, and the resulting extract files can be concatenated as input to one ROLLFORWARD job.
COMPLETE and ALL with STOP time considerations
If the COMPLETE option or the ALL option with a STOP time is specified, the resulting extract file will not contain any information for run units that were active at the end of the archive file. Therefore, an extract file created with these options should never be used as input with extract files created from subsequent archive journals. Use these options only if you intend this extract file to be the last in a series of extract files that will be used as input to a ROLLFORWARD.
Multiple output extract files from one archive journal
To create multiple extract files from one archive journal (one for each database segment, for instance) EXTRACT JOURNAL must be run multiple times, once for each group of database areas or files that you may wish to recover separately.
Since ROLLFORWARD has the ability to select images from an input extract file by area or file, you do not need an extract file for each area or file. For example, you can create an extract file for each critical database segment, and if a recovery is needed for one file in that segment, you can use ROLLFORWARD to recover just that file from the segment extract file. This speeds recovery while minimizing the number of extract tapes you need to maintain.
KSDS native VSAM records
EXTRACT JOURNAL does not support KSDS native VSAM records.
Controlling the starting point of the extract operation
If no START AT parameter is specified, the extract operation starts with the first journal image on the input archive file(s). If START AT is specified, the extract operation starts with the first checkpoint record (BGIN, COMT, ENDJ, ABRT, or CKPT) whose timestamp is on or after the specified time.
Specifying a start time may save recovery time and also circumvent issues associated with aborted recovery units that span the start of the input file. It further enables the extract operation to begin processing at a quiesce point that does not correspond with a journal file boundary.
If START AT is specified, quiesce point verification is always performed even if VERIFY QUIESCE NONE is specified. Limited verification will be performed unless full verification is specifically requested.
Quiesce point verification
A quiesce point is a point in time during which there is no update activity for some portion of a database. To perform a correct recovery, you must begin the recovery operation at a quiesce point for the portion of the database being recovered. To assist in this effort, the extract journal utility provides three levels of quiesce point verification: none, limited, and full.
None means that no quiesce point verification is performed. It is appropriate for extract operations whose input is not expected to coincide with a quiesce point. For example, if extracting journal information incrementally, then quiesce point verification could be used when processing the archive files produced immediately after a quiesce operation, but must not be used when processing subsequent archive files.
The limited and full options both enable quiesce point verification. They do this by checking for the existence of spanned recovery units that are recovery units that are active at the start of the extract operation. A recovery unit is represented by journal images starting with a BGIN or COMT checkpoint and ending with a COMT, ENDJ, or ABRT checkpoint. If a spanned recovery unit updates the specified portion of the database, then the extract operation is not starting at a quiesce point. If this situation is detected and either limited or full quiesce point verification is in effect, the extract operation will terminate with an error.
However, it is not always possible to know whether a spanned recovery unit affects the specified portion of the database or not. If the initial BGIN or COMT checkpoint record for a recovery unit is not contained on the archive file being processed, then it is not possible to determine whether it updated the specified portion of the database. Such a recovery unit is referred to as an indoubt recovery unit.
The time line illustrates what is meant by an indoubt recovery unit. The journal images for recovery unit
R
are written to the journal file at the times shown. If the archive file includes images starting at T1, then
R
is not an indoubt recovery unit because the archive file contains all journal images written for
R
since its inception. Similarly, if the archive file starts at time T3,
R
is not an indoubt recovery unit, because the archive file contains no images for
R
whatsoever. However, if the archive file starts at time T2, then
R
is an indoubt recovery unit, since the archive file does not contain all journal images written since its inception.
If an indoubt recovery unit does not span the start of the rollforward operation, its existence doesn't matter. But if an indoubt recovery unit is also a spanned recovery unit, then the extract operation might not be starting at a quiesce point.
The action taken if an indoubt spanned recovery unit is encountered depends on whether limited or full quiesce point verification is in effect. Under full verification, the extract operation will terminate with an error. Under limited verification, a warning message will be issued identifying the recovery unit, but processing will continue. Warning messages produced under limited verification should be examined to ensure that the identified recovery units in fact did not affect the specified portion of the database. If there is any doubt, the PRINT JOURNAL utility statement should be used to gain more information about the indoubt recovery units. If after researching the situation, it is found that an indoubt recovery unit did update the specified portion of the database, the resulting extract file must not be used for recovery purposes. You must locate a quiesce point corresponding to a backup of the specified portion of the database and begin the extract operation from that point.
When to use full or limited verification
Full quiesce point verification should only be used if you expect that no indoubt spanned recovery units are active at the starting point of the extract operation. The only way to guarantee this is to process the archive files that were created immediately following a quiesce of update activity across all areas. One way to establish such a quiesce point is to shut down a central version. Another way is to use the DCMT QUIESCE command and specify a DBNAME that includes every area in the DMCL.
Limited quiesce point verification can be used when processing the archive files produced immediately following a quiesce operation for the portion of the database for which the extract is being performed. One way to do this is to use the DCMT QUIESCE SEGMENT command to quiesce a segment and then use the limited quiesce point verification when extracting records for that segment.
EXTRACT JOURNAL and Distributed Transactions
EXTRACT JOURNAL reports on distributed transactions and supports the use of an input section recovery control file. The control file is used to complete InDoubt distributed transactions unless ALL is specified. For more information, see JCL Considerations as well as Common Facilities for Distributed Transactions.
For considerations associated with distributed transactions during recovery operations, see the
CA IDMS Database Administration Section
JCL Considerations
When you submit an EXTRACT JOURNAL statement through the batch command facility, in addition to the standard JCL required for the batch command facility, you must also include statements to define:
  • SYS001 to point to the input archived journal file.
  • SYS002 or the DDname specified in the JCL to point to the output extract file. The output extract file is a standard variable length record file. Specify a block size that is at least as large as the block size of the input journal.
  • Any sort work files needed by your local sort.
To use a section recovery input control file, include a CTRLIN file definition or DD statement in the IDMSBCF execution JCL. To use a section recovery output control file, include a CTRLOUT file definition or DD statement in the IDMSBCF execution JCL. The format of both of these files is fixed block with a record length of 80.
For more information about the generic JCL used to execute the batch command facility, see the section for your operating system in this section.
Example
The following statement directs the EXTRACT JOURNAL utility to create an extract file from all after images in the EMPDEMO segment. The default ddname of SYS002 is used.
extract journal for segment empdemo all;
Sample Output
The first listing shown next, was generated after submitting the sample EXTRACT JOURNAL statement in the previous example. The extract file was then used as input to the ROLLFORWARD utility to restore the EMPDEMO segment. The listing from the ROLLFORWARD utility is presented after the EXTRACT JOURNAL listing.
EXTRACT JOURNAL listing
IDMSBCF                                              IDMS Batch Command Facility                            mm/dd/yy   PAGE 1   EXTRACT JOURNAL FOR      AREA EMPDEMO.EMP-DEMO-REGION,           EMPDEMO.INS-DEMO-REGION,           EMPDEMO.ORG-DEMO-REGION      ALL      TO SYS002      STOP AT 'yyyy-mm-dd-hh.mm.ss.ffffff'      ; ROLLFORWARD STARTED yyyy-mm-dd-hh.mm.ss.ffffff  RU_ID  00000001  PGM_ID  EMPLOAD   QUIESCE LEVELS 01  UPD 00   BGIN  yyyy-mm-dd-hh.mm.ss.ffffff  RU_ID  00000001  PGM_ID  EMPLOAD   QUIESCE LEVELS 00  UPD 00   ENDJ  yyyy-mm-dd-hh.mm.ss.ffffff IMAGES SELECTED FOR AREA EMPDEMO.EMP-DEMO-REGION              1,034 IMAGES SELECTED FOR AREA EMPDEMO.INS-DEMO-REGION                196 IMAGES SELECTED FOR AREA EMPDEMO.ORG-DEMO-REGION                587 TOTAL IMAGES WRITTEN                1817 JOURNAL INPUT COUNTS: BLOCK COUNT       705  BACKWARD        0 RECORD COUNT     5526  BACKWARD        0 Status = 0 AutoCommit will COMMIT transaction Command Facility ended with no errors or warnings
Listing from ROLLFORWARD
IDMSBCF                                              IDMS Batch Command Facility                            mm/dd/yy   PAGE 1   ROLLFORWARD       SEGMENT EMPDEMO   FROM EXTRACT SYS002   ; ROLLFORWARD STARTED yyyy-mm-dd-hh.mm.ss.ffffff IMAGE FILE CREATED ON yyyy-mm-dd-hh.mm.ss.ffffff   EXTRACT FILE IMAGES CREATED FROM yyyy-mm-dd-hh.mm.ss.ffffff TO yyyy-mm-dd-hh.mm.ss.ffffff RECORDS RESTORED TO AREA EMPDEMO.EMP-DEMO-REGION              1,034 RECORDS RESTORED TO AREA EMPDEMO.INS-DEMO-REGION                196 RECORDS RESTORED TO AREA EMPDEMO.ORG-DEMO-REGION                587 TOTAL RECORDS RESTORED              1817 JOURNAL IMAGES READ    1,818 Status = 0 IDMSBCF                                              IDMS Batch Command Facility                            mm/dd/yy   PAGE 2 AutoCommit will COMMIT transaction Command Facility ended with no errors or warnings