File Definition and Maintenance

ce18-18-0-11
This list describes the
Endevor
data sets:
  • Master Control File
    Definitions of stages, systems, subsystems, element types, and elements. This file is accessed and updated by
    Endevor
    to perform source and output management, and to handle other miscellaneous services. There is one Master Control File (MCF) for each stage at a site. The Master Control Files are defined as a function of installation.
  • Package data set
    Package information for all environments defined for the site.
  • Base and delta libraries
    Source statements for elements, including processors. The base library contains the source as originally added to
    Endevor
    . The delta library contains the changes made to the elements. These libraries are specified separately to each element type definition, but can be shared by multiple element types. The libraries can be shared across systems and stages, but make sure that the defined record length for each library is adequate to store the element source from all systems/stages that share the library. At a minimum, you must have one library to store base and delta members in each environment.
    If you use
    Endevor
    ACM, the component base and delta are also stored in these same libraries. The delta library requires a logical record length of at least 259. Each base or delta library can be an OS partitioned data set (PDS or PDS/E), a
    Panvalet
    data set, a
    Librarian
    data set, or a self-reorganizing ELIB data set.
  • Source output library
    Latest full source form of each element, created during output management. This is an optional library defined to each element type (although the same library can be shared across element types). This library is designed for use with COBOL copybooks, assembler macros, or JCL procedures that are copied elsewhere (and therefore have to be available in full source form). It can be used for any type element however.
    The source output library can be an OS PDS or PDS/E, or a
    Panvalet
    or
    Librarian
    data set.
  • Endevor
    listing library
    Listings output by the
    Endevor
    CONLIST utility or by the type PROCESS generate processor (GPPROCSS). For both listing purposes, this library must be specific to a stage, but can be shared across systems. This library can be an OS PDS or PDS/E or ELIB data set.
  • Processor load library
    Load module form of each processor defined to
    Endevor
    , as output by the type PROCESS generate processor, GPPROCSS. This library is specific to a stage, but can be shared across systems. This library must be an OS load library (RECFM=U).
In addition to the libraries referenced, you may also have INCLUDE libraries and user libraries (copy libraries, macro libraries, JCL libraries) defined to processors. These libraries are not specific to
Endevor
, and therefore are not described here.
How to Define the
Endevor
Files
You establish your
Endevor
files during installation. The following instructions explain how to set up additional base and delta libraries, processor listing libraries, processor load libraries, and source output libraries, as well as how to monitor and maintain the existing libraries.
If you are currently using OS PDS or PDS/Es for base, delta, and/or listing libraries, consider converting these PDS or PDS/Es to ELIB self-reorganizing data sets.
ELIB Data Sets
ELIB data sets offer several performance advantages over OS PDS. ELIB data sets:
  • Automatically reorganize member space as members are rewritten or deleted, thereby eliminating the need to compress the data set.
  • Exploit 31-bit storage for VSAM-organized data sets, thereby reducing 24-bit storage contention.
  • Expand directories and data sets automatically.
  • Provide improved directory processing.
  • Maintain additional statistical information about the member size.
These features eliminate most of the growth and compress problems involved with managing PDS or PDS/Es. ELIB provides faster support for add, update, and delete activities due to its advanced directory processing techniques.
For more information about ELIB data sets, see
Setting Up ELIB Data Sets
.
How to Define Base and Delta Libraries
Base and delta libraries can be ELIB (ELIB) data sets, partitioned data sets (PDS or PDS/E), or
Panvalet
or
Librarian
data sets. Depending upon the delta format chosen, each base/delta library set can be any combination of these file types.
For example, if you use reverse deltas, a regular PDS or PDS/E to be used to store element base, and an ELIB dataset to store deltas. Base libraries can be USS directories.
Base and Delta Library Space Requirements
Space requirements for base libraries are a function of the number of elements (members) to be stored, the number of source lines per element (for base libraries), the volatility of the elements (for delta libraries that is, the number and extent of expected changes), and the library management facility in use.
Allocate a New PDS or PDS/E
  1. To allocate a new PDS or PDS/E, use ISPF/PDF option
    3
    (Utilities), option
    2
    (Data sets), or any suitable IBM utility (such as IEFBR14).
  2. Specify the following DCB, assigning a block size appropriate to your disk device:
    DCB=(RECFM=VB,LRECL=
    record length
    ,BLKSIZE=
    block-size
    )
    Where record length is the maximum record length that you anticipate storing in the PDS or PDS/E plus the constant 4. Block-size is a number at least 4 greater than the LRECL length.
    When specifying the number of directory blocks in each library, you have one directory block for every four elements. For efficiency, directory blocks should be allocated in increments of 45 for a 3390-type device (whatever number can fit on a single track, if you are using another type of device).
    To calculate this number, divide the estimated number of elements (members) to be stored in the library by 4. Round up to an even multiple of 45 (assuming a 3390-type device). The number should be the same for the base and delta libraries.
If the Automated Configuration Manager facility (
Endevor
ACM) is installed at your site, increase the file size by 20 percent and double the number of directory blocks.
Allocate a New
Panvalet
or
Librarian
Data Set
Allocate and Initialize a New ELIB Data Set
To allocate and initialize a new ELIB data set, see
Setting Up ELIB Data Sets
.
Define an
Endevor
Listing Library
Space requirements for an
Endevor
listing library are a function of the number of elements added to the system(s) with which the library is associated (by CONLIST or type PROCESS), and the number of lines in each listing.
To allocate a new PDS or PDS/E, use ISPF/PDF option
3
(Utilities), option
2
(Data sets), or any suitable IBM utility (such as IEFBR14). Specify the following DCB, assigning a block size appropriate to your disk device:
DCB=(RECFM=VBA,LRECL=259,BLKSIZE=
block-size
)
Specify the number of directory blocks needed. There is one directory block for every four listing members. For efficiency, directory blocks are allocated in increments of 45 (or whatever number can fit on a single track, if you are using a device other than a 3390-type device).
To calculate this number, divide the expected number of listings by four and round up to an even multiple of 45 (for a 3390-type device).
For more information about allocating a new ELIB data set, see
Setting Up ELIB Data Sets
.
Define a Processor Load Library
Space requirements for this library are a function of the number of processors added to the system(s) with which the library is associated, and the number of lines in each processor.
To allocate the library, use ISPF/PDF option
3
(Utilities), option
2
(Data sets), or any suitable IBM utility (for example, IEFBR14). Specify the following DCB, assigning a block size appropriate to your disk device:
DCB=(RECFM=U,BLKSIZE=
block-size
)
Specify the number of directory blocks needed. There is one directory block for every four processors. For efficiency, directory blocks are allocated in increments of 45 (or whatever number can fit on a single track, if you are using a device other than a 3390-type device).
To calculate this number, divide the expected number of processors by four, then round up to an even multiple of 45 (for a 3390-type device).
Define a Source Output Library or USS Directory
Each source output library can be a USS directory, a partitioned data set (PDS or PDS/E), or a
Panvalet
or
Librarian
library. If it is a PDS or PDS/E, it can have fixed or variable-length records.
Follow these steps:
  1. Use ISPF/PDF option
    3
    (Utilities), option
    2
    (Data sets), or any suitable IBM utility (such as IEFBR14).
  2. Specify the following DCB, assigning a block size appropriate to your disk device:
    DCB=(RECFM=FB,LRECL=80,BLKSIZE=
    block-size
    ) (fixed records)
    or
    DCB=(RECFM=VB,LRECL=
    rec-len
    ,BLKSIZE=
    block-size
    )(variable records)
    rec-len
    is the maximum record length (as specified when defining the element type(s) that use this library), plus 4.
    You must specify the number of directory blocks needed. There is one directory block for every four source modules. For efficiency, directory blocks are allocated in increments of 45 (or whatever number can fit on a single track, if you are using a device other than a 3390-type device).
    To calculate this number, divide the expected number of source modules by four, then round up to an even multiple of 45 (for a 3390-type device).
How to Maintain Files
The
Endevor
administrator must monitor the
Endevor
files regularly, to ensure that they have adequate space. This section describes some of the standard tools you, as the
Endevor
administrator can use to monitor and maintain the files.
Monitor Space Utilization
The Master Control File and Package data sets are VSAM files, and should be maintained using the standard IBM VSAM maintenance utility IDCAMS. To obtain file utilization statistics, run the utility with the LISTCAT command.
Monitor your
Endevor
libraries regularly, as explained in the following:
  • OS PDS or PDS/E libraries
    Use ISPF/PDF, option
    3
    (Utilities), option
    2
    (Data sets) to display space utilization statistics. OS PDS or PDS/E libraries include the
    Endevor
    listing libraries and processor load library, and may include the base and delta libraries and the source output library.
  • Librarian
    or
    Panvalet
    libraries
    The base library, delta library, and source output library may be
    Librarian
    or
    Panvalet
    files.
  • ELIB data sets
    Run the BC1PNLST utility program. ELIB data sets can be used for the base, delta, and/or listing libraries.
    For more information about the BC1PNLST utility, see
    Setting Up ELIB Data Sets
    .
How to Expand or Compress a File
If a file becomes full, compress or expand the file, as appropriate. This section explains how to do this for the Master Control File, Package data sets, PDS or PDS/E libraries,
Librarian
and
Panvalet
libraries and ELIB data sets.
Expand or Compress the Master Control File
For the Master Control File (MCF), run the job supplied as member BC1JRMCF in your installation JCL library. Use this job to expand the files, to periodically clean up control interval splits, or both. This job processes the Stage 1 and Stage 2 files. When expanding an MCF, you can modify the procedure as necessary.
Expand or Compress Package Data Sets
For package data sets, run the job supplied as member BC1JRPKG in your installation JCL library. Use this job to expand the data set, to periodically clean up control interval splits, or both. When expanding the data set, you can modify the job as necessary.
Reorganize Element Catalog and Eindex Files
For Element Catalog and Eindex files, run the job supplied as member BC1JRCAT in your installation JCL library. Use this job to reorganize the Element Catalog and Eindex files.
Expand or Compress PDS or PDS/E Libraries
For partitioned data set libraries, run a batch job using the IBM IEBCOPY utility, such as that shown next, to perform the compression. The first step of the job should back up the data set in the event of a subsequent problem.
//BACKUP EXEC PGM=IEBCOPY //SYSPRINT DDSYSOUT=* //DFILE DDDSN=library-dsn,DISP=OLD //TFILE DDDSN=seq-backup-dsn,DISP=(NEW,CATLG),UNIT=TAPE //SYSIN DD* COPY INDD=DFILE,OUTDD=TFILE /* //COMPRESS EXEC PGM=IEBCOPY,COND=(0,NE,BACKUP) //SYSPRINT DDSYSOUT=* //DFILE DDDSN=library-dsn,DISP=OLD //SYSIN DD* COPY INDD=DFILE,OUTDD=DFILE /*
Expand or Compress
Librarian
and
Panvalet
Libraries
Librarian
or
Panvalet
data sets are automatically compressed each time a member is stored or updated and therefore, compression is unnecessary. Should a library become full, follow the appropriate
Librarian
or
Panvalet
procedures to expand the file.
Expand or Compress ELIB Data Sets
ELIB data sets are automatically maintained each time a member is stored or updated and therefore, compression is unnecessary. In addition, ELIB data sets can automatically expand into secondary extents.
For more information about changing the secondary expansion quantity or the directory size, see
Setting Up ELIB Data Sets
.
How to Back Up
Endevor
You should back up the
Endevor
files regularly. Back up all files associated with a stage together, when no update processing is occurring against the stage. Define each data set being backed up with DISP=OLD, to ensure that no update processing can occur against the data set during backup.
Each
Endevor
file is classified as critical or non-critical. Critical files, if lost, can only be rebuilt from a backup. Critical files are the Master Control File, base library, and delta library. Non-critical files include the source output library, processor listing library, and processor load library. Non-critical files can be rebuilt by processing the elements. If the source output library is lost, for example, the source can be recreated from the base and delta libraries.
The following describes how to back up various
Endevor
files:
  • The Master Control File
    Use the IDCAMS utility with the REPRO command.
  • A PDS or PDS/E
    Use a standard IBM utility (for example, IEBCOPY), or any other appropriate utility in use at your site.
  • A
    Panvalet
    or
    Librarian
    file
    See the appropriate documentation.
Back Up VSAM ELIB Data Sets
To back up ELIB VSAM data sets, use the IDCAMS utility and supply the appropriate space and DCB information, and is located in the member BC1JELIB is supplied in your
iprfx.iqual
JCL library for this purpose. You can also use any other appropriate utility in use at your site.
Back Up BDAM ELIB Data Sets
To back up an ELIB BDAM data set, use the standard IBM IEBGENER utility or any other appropriate utility in use at your site.
How to Back Up Using the Unload, Reload, and Validate Utility
If you do not want to use the standard IBM backup utility, you can use the
Endevor
Unload, Reload, and Validate utility. This utility provides a backup, reload, and validation mechanism for
Endevor
VSAM files (Master Control File and package data set) and base/delta libraries.
For more information about using this utility, see
Unload, Reload, and Validate
.
How
Endevor
Recovery Works
If a critical file is lost or its integrity compromised, restore all critical files, including the Master Control File, base library, and delta library. First restore the files using the latest backup. Then reapply any updates made since the last backup.
Keep the source version of all changes made to critical files until a backup can be made. Otherwise, you cannot reapply these updates.
The Footprint Exception Report (CONRPT83) identifies those members of an output library that are out of sync with the MCF, and can be used to identify the members that must be added/updated in the Master Control File. If allocated, the source output library can be used to assist in the recovery. In the event that source code is lost prior to taking a backup of the critical files, the source output library still contains a current copy of the source.
If a non-critical file is lost or its integrity compromised:
  • Rebuild the file by reprocessing the lost elements. Again, non-critical files include the source output library,
    Endevor
    listing libraries, and processor load library.
  • If you have a backup copy of the file that is in sync with the critical files, you can recover directly from the backup. If you do this, however, first ensure that the restored file matches the information contained in the current MCF, base library, and delta library exactly.
How to Recover Using the Unload, Reload, and Validate Utility
You can use the
Endevor
Unload, Reload, and Validate utility for recovery. This utility provides a backup, reload, and validation mechanism for
Endevor
VSAM files (Master Control File and package data set) and base/delta libraries.