Chapter 5: High Availability and Replication

Overview

RSF supports replication of libraries, IFS directories, user profiles and other system objects for high availability and recovery.


Menu Options

Replication functions are collected on two High Availability menus.  You can access the source machine menu by selecting option 40 from the main RSF menu followed by option 7 from the Advanced Functions menu, or by simply running the command

             GO  RSF/RSFHA.

For more information about any of the commands accessed from this menu, prompt the command and press F1 to view the online help text.

RSFHA                  RSF - High Availability 
                        Source Machine Tasks

Connections                          Stop Synchronizing                   
1. Define outbound connections      
30. Libraries, system info, IFS
2. Start server function             31. Spooled files 32. Message queues
3. End server function
                                     Monitor                              
Set Sync Attributes:                
40. Work with sync attributes
10. Library      12. IFS             41. View log      44. Waiting jobs
11. System info  13. From list       42. Purge log     45. Job logs
                                     43. Active jobs
Start Synchronizing                   
20. Custom       24. From list       Other                                
21. Libraries    25. From Sync Attr  80. Defaults      83. Integrity tools
22. System info  26. Spooled files   81. Edit source   88. Target menu
23. IFS          27. Message queues  82. Tools menu    89.
Role swap

Selection or command
===>                                                                     

 

 

Option 1.

This option is used to define the connection between the source and the target machine.  This is the same as option 1 on the main RSF menu, repeated here for convenience.  Use this option to define an RSF Server Directory Entry for the target machine.  See Chapter 4: Making Connections for more information about defining an RSF connection.

Option 2.

Use this option to start  the RSF server function.  The server function must be started on the each machine in order for other machines to initiate contact.

This is the same as option 21 on the main RSF menu, repeated here for convenience.

Option 3.

Use this option to end  the RSF server function on this machine. 

This is the same as option 22 on the main RSF menu, repeated here for convenience.

Options 10 - 13.

Use these options to set synchronization attributes.  The commands for changing synchronization attributes can also be accessed from the Work With Sync Attributes display (option 40).

Among other things, changing synchronization attributes lets you establish:

 

                 Change Library Sync Attributes (CHGRSFSA)

Type choices, press Enter.

Library  . . . . . . . . . . . .              Name
To library . . . . . . . . . . . *FROMLIB     *FROMLIB Name, *FROMLIB
Server ID  . . . . . . . . . . .              Name
Journal  . . . . . . . . . . . . *NONE        *SAME Name, *SAME, *NONE
   Library . . . . . . . . . . .              Name, *FROMLIB
Remote journal . . . . . . . . .              Name, *SAME, *NONE, *JRN
   Library . . . . . . . . . . .              Name, RMTJOURNAL
RSF manages the journal  . . . . *YES         *SAME *YES, *NO, *SAME
Change receiver every  . . . . . *DAY         *INTERVAL, *DAY, *SYS, *SAME
Days to keep receivers . . . . . *SYNC        Number, *SYNC, *SAME
End jrn for excluded objects . . *YES         *SAME *YES, *NO, *SAME
Fix obj using wrong journal  . . *YES         *SAME *YES, *NO, *SAME
Apply journal entries by . . . . *KEY         *SAME *KEY, *RRN, *SAME
Refresh on journal apply error   *YES         *SAME *YES, *NO, *SAME
Sync object authorities  . . . . *NO          *SAME *YES, *NO, *SAME
Objects to omit:                  
  Object . . . . . . . . . . . . *NONE        *SAME Name, generic*, *NONE...
  Object type  . . . . . . . . .              *ALL, *ALRTBL, *BNDDIR...
           + for more values     


                                                               More...

Use the Change Library Sync Attributes (CHGRSFSA) command (option 10) to determine how a library is synchronized.  A view of the CHGRSFSA command prompt is shown above.

Use the Change System Sync Attributes (CHGRSFSSA) command (option 11) to determine how system objects are synchronized.

Use the Change IFS Sync Attributes (CHGRSFISA) command (option 12) to determine how IFS directories are synchronized.

Use the Set Sync Attributes For a List (SETRSFSA) command (option 13) to set synchronization attributes for many libraries and system object types at once.  Not all attributes can be set with this option, but many of the most common ones can.  This option helps you get started when you have many libraries you would like to define for synchronization.  Source for the command and command processing program are included in RSFTOOLS.

Options 20 - 26.

Use these options to begin replicating libraries, IFS directories, user profiles and so on.

Use the Start Synchronization (STRSYNCRSF) command (option 20)  to start all replication tasks on your system in a consistent, customized way.  Once you are familiar with RSF-HA, this is the option you will use most often to start synchronization.  See Create a Synchronization Start Program in the Where to Begin section for more information.

                    Synchronize Libraries (SYNCLIBRSF)

Type choices, press Enter.

Library specifications:            
  From library . . . . . . . . .              Name, generic*, *ALLUSR
  To library . . . . . . . . . .              *FROMLIB Name, *FROMLIB
  Restore to ASP device  . . . .              *SAVASPDEV Name, *SAVASPDEV
  Restore to ASP number  . . . .              *SAVASP 1-32, *SAVASP
               + for more values 
Server ID  . . . . . . . . . . .              Name
Target release . . . . . . . . . *CURRENT     Character value, *TARGET...
Data compression . . . . . . . . *YES         *YES, *NO
Run-time compression . . . . . . *NONE        *NONE, *BASIC, *MAX, *SERVER
Repeat every:
  Interval . . . . . . . . . . . 0            Number, *ATTR
  Units  . . . . . . . . . . . . *HOURS       *HOURS, *MINUTES
Job description . . . . . . . .  *NONE        Name, *NONE, *USRPRF
  Library . . . . . . . . . . .               Name, *LIBL, *CURLIB
Job name . . . . . . . . . . . . SYNCLIB      Name
                                                                   More...

Use the Synchronize Libraries (SYNCLIBRSF) command (option 21) to begin replication of one or more libraries.  A view of the SYNCLIBRSF command prompt is shown above.

Use the Synchronize System Info (SYNCSYSRSF) command (option 22) to begin replicating user profiles, authorization lists and other system information.

Use the Synchronize IFS Directories (SYNCIFSRSF) command (option 23) to begin replicating IFS directories.

Use the Synchronize List of Items (SYNCLSTRSF) command (option 24) to begin replicating a list of items.  Source for the command and command processing program are included in RSFTOOLS.

Use the Synchronize From Attributes (SYNCATRRSF) command (option 25) to start replication for some or all of the
libraries libraries and IFS directories defined on your Work With Sync Attributes (WRKRSFSA) display.

Use the Start Monitoring Output Queue (STRMONOUTQ) command (option 26) to begin replicating spooled files.

Use the Start Message Queue Monitor (STRRSFMSGM) command (option 27) to begin replicating message queues.

Options 30 - 32.

Use these options to end  replication for libraries, IFS directories, user profiles and so on.

Option 40.

This is an option you will use often.  With this option you access the Work With Sync Attributes display.  You can choose to view all libraries, IFS directories and system information currently being synchronized or those items not being synchronized.  On  the display you can see the last synchronization date for each item.  You can also easily edit synchronization attributes, view error logs, submit  synchronization jobs and more.

10/29/10              Work With Synchronization Attributes
15:25:00
                                           Position to . . . . .              

Type options, press Enter.
  2=Attributes  4=Delete  5=Display log  8=Sync  10=DSPJRN  12=WRKJRNA


                                       ---- Last Sync ----  --- Last Error ----
Opt From        Server     To          Date       Time      Date       Time
    *AUTL       BOB        QSYS        2009/10/01 21:47:23
    *NETA       BOB        QSYS        2009/10/01 21:47:19
    *SYSVAL     BOB        QSYS       
2009/10/01 21:47:15
    *USRPRF     BOB        QSYS        2009/10/01 21:47:09
    /qibm/user  BOB        rdata/icss  2009/10/01 12:56:29
    /www/webse  BOB        /webserver  2009/10/01 12:56:32
 
  INVEN       LOCAL      INVEN2
    INVEN       LOOPBACK   INVEN3
    RJTEST      LOOPBACK   RTTEST2
    ELF         BOB        ELF         2009/10/01 21:40:58
    PRODINFO    BOB        PRODINFO    2009/10/01 21:40:55
    ACCTAPP     BOB        ACCTAPP     2009/10/15 10:46:23

                                                                     More...
F3=Exit F4=Prompt F5=Refresh F6=Create F23=More options F24=More keys

 

Option 41.

Use this option  to display the synchronization error log.  An entry is placed in the log any time an error is detected synchronizing libraries, IFS directories or system information.

The Display Synchronization Log (DSPRSFSL) command is also accessible from the Work With Sync Attributes display.

Option 42.

Use this option to delete selected entries from the synchronization error log.

Option 43.

Use this option to view active jobs in subsystem RSFHA..

Option 44.

Use this option to view jobs waiting on job queue RSFHA..

Option 45.

Use this option to work with any RSF related job logs.

Option 80.

Use this option to change global RSF defaults.  The Change RSF Defaults (CHGRSFDFT) command prompt is displayed.

Option 81.

Use this option to work with source for user-modified programs in library RSFUSER.

Option 82.

Use this option to display the RSFTOOLS menu.

Option 83.

Use this option to display the Integrity Tools (RSFINT) menu.  From this menu you can compare libraries and IFS directories, archive/refresh libraries and directories, and more.  Some of the integrity functions can also be accessed from the Work With Sync Attributes display (option 40.)

Option 88.

Use this option to display the target machine HA menu, RSFHAT.

Option 89.

Use this option to change the role of this machine.  See the Role Swap section for more information.

 

Target Machine Options

You can access the backup machine HA menu by selecting option 80 from the from the source machine HA menu, or by simply running the command

             GO  RSF/RSFHAT.

For more information about any of the commands accessed from this menu, prompt the command and press F1 to view the online help text.

RSFHAT                   RSF - High Availability
                          Target Machine Tasks

Connections                          Other                               
 1. Options for inbound connections  80. Defaults
 2. Start server function            88. Source menu
 3. End server function             
89. Role swap

Set Sync Attributes:                  
10. Change unique key mapping 

Monitor                               
43. Active jobs 
45. Job logs  
46. History log 
 



Selection or command
===>                                                                      

 

 

Option 1.

Use this option to set the rights and options that apply when the source machine contacts this (the target) machine.

This is the same as option 23 on the main RSF menu, repeated here for convenience.

Option 2.

Use this option to start  the RSF server function.  The server function must be started on the target machine in order for replication to begin.

This is the same as option 21 on the main RSF menu, repeated here for convenience.

Option 3.

Use this option to end  the RSF server function on the target machine. 

This is the same as option 22 on the main RSF menu, repeated here for convenience.

Option 10.

Use this option to invoke the Change Unique Key Association  (CHGRSFUKEY) command.  With the CHGRSFUKEY command you can:

Option 43.

Use this option to view active jobs for user RSFSRV.

Option 45.

Use this option to work with any RSF related job logs.

Option 46.

Use this option to view any RSF entries in the system history log.  RSF history log entries are generated for each journal apply error as well as for apply jobs that end in error.

Option 88.

Use this option to access the source machine HA menu, RSFHA.

Option 89.

Use this option to change the role of this machine.  See the Role Swap section for more information.

 


Where to Begin

This section outlines the basic steps required to begin replicating from one system to another.

1. Create a Library for User Objects and a new Subsystem

As you setup RSF-HA, you will be prompted to create some customized objects which help make managing your replication environment easier.  Now is a good time to create a new library in which to store these objects.  We recommend calling this library RSFUSER.

In addition, you will find it useful to have a separate subsystem, job queue and job description for use by RSF-HA.

Tip: To accomplish all of these tasks, simply use the Create Sync Subsystem (CRTSYNCSBS) command, which which can be found in library RSFTOOLS.  The command shown below has the recommended values filled in.

CRTSYNCSBS LIB(RSFUSER) SHRPOOL(*SHRPOOL1) CRTLIB(*YES)

See option 30 on the RSFTOOLS menu.

2. Collect Some Information

How many libraries on the production system need to be replicated?
How big are they?
Which libraries need to be synchronized up to the minute and which less frequently?
Are there any cross-library dependencies?
Are some objects already being journaled?
What IFS directories need to be replicated?

Knowing the answers to these questions makes setting up replication easier.  An option on the RSFTOOLS menu can help you:

Type

GO RSFTOOLS/RSFTOOLS

to display the menu.  Then select option 36 ("Display all") to run the necessary reports in batch.

If you plan to use remote journaling, see the Remote Journaling topic in the Things to Consider section for additional preparation tasks.

3. Check Available Disk Space

Use the Work with System Status (WRKSYSSTS) command on the source machine to see how much disk space is being used currently.  The display should look similar to the following:

                    Work with System Status                      BUG0710
                                                        12/18/10 11:05:04
% CPU used . . . . . . . :      7.6    Auxiliary storage:
% DB capability  . . . . :       .0      System ASP . . . . . . :
70.56 G
Elapsed time . . . . . . : 00:00:01      % system ASP used  . . :
42.1758
Jobs in system . . . . . :     6402      Total  . . . . . . . . : 70.56 G
% perm addresses . . . . :     .013      Current unprotect used :  2873 M
% temp addresses . . . . :     .059      Maximum unprotect  . . :  3535 M


Type changes (if allowed), press Enter.

System     Pool     Reserved    Max  -----DB----- ---Non-DB---
 Pool    Size (M)   Size (M)  Active Fault  Pages Fault  Pages
  1      139.92      70.81     +++++    .0     .0    .8     .8
  2      352.23       2.04        23    .0     .0    .0     .0
  3      175.47        .00        12    .0     .0   7.1    7.1
  4         .25        .00         5    .0     .0    .0     .0
  5      224.32        .00         5    .0     .0    .0     .0

                                                                  Bottom
Command
===>                                                              

Multiply (0.01 * %_system_ASP_used * System_ASP) to determine the disk space currently used on the production machine.  In this example, in round numbers, (.01 * 42 * 71) = 30 GB used.

Then, check on the target machine to be sure you have this much disk space available , plus a fair amount more to allow for expansion and storage of temporary objects such as journal receivers.

4. Establish a Connection

Follow the steps outlined in Chapter 4: Making Connections to establish a connection between the production and backup machines.  Then, set up a connection going the other way as well--from backup to production.

5. Synchronize User Profiles

Before synchronizing any other objects, you should ensure that all user profiles defined on the production machine also exist on the backup machine.  This will avoid operating system authority errors when synchronizing and restoring objects to the backup machine.

The easiest way to accomplish this is to synchronize user profiles from the production to the backup machine.  To begin, define user profile replication using the Change System Sync Attributes (CHGRSFSSA) command, or RSFHA menu option 11, as follows:

CHGRSFSSA SERVER(your_server_ID) TYPE(*USRPRF) OPTION(*OMIT) ITEMS(Q*) DISABLEPRF(*NO)

As specified above, system profiles are omitted, and no user profiles are disabled on the target machine.  (To automatically disable profiles on the target machine, specify DISABLEPRF(*YES) instead.)

Next, run the Work With Sync Attributes (WRKRSFSA) command, or RSFHA menu option 40, to display the list of items ready to be synchronized. 

At the Work With Sync Attributes display, press F13 to set some user defaults.  Ensure that *YES is specified for "Run in batch" and RSFUSER/RSFHA is specified for "Job description".

Then, key option 8 beside the *USRPRF entry and press Enter to launch user profile replication.

Important Note: When synchronizing user profiles for the first time, it is common to see some errors.  Often, objects needed by one or more user profiles have not been replicated to the target system yet.  These errors can safely be ignored at this time.  If errors are detected, do a quick visual check to be sure all profiles from the source machine were copied to the target and then run the following command to establish a synchronization check point for user profiles:

SYNCSYSRSF TYPE(*USRPRF) SERVER(your_server_ID) FULLSAVE(*MANUAL)

Here's a summary of the tasks involved in this step

6. Create a Journal Library.

If you will be synchronizing database files at the record level or IFS objects at the byte level, journals will be needed to capture the detail changes.  RSF can create the journals for you automatically or you can tell RSF to use existing journals.

Unless you must use pre-existing journals for replication we recommend that you:

Note: Remote journaling uses TCP ports 446 (DRDA),  and 447 (DDM) and 3777 (remote journaling proper).  If you intend to use remote journaling and your source and target machines are separated by a firewall, be sure to open ports 446, 447 and 3777.

In addition, you should use the Change DDM TCP/IP Attributes (CHGDDMTCPA) command on the target machine to change the "Password Required" option to *NO, as the *YES option does not work in most environments.

7. Categorize Your Libraries

It's helpful to divide your libraries in to groups as follows.  Libraries in each group will have similar synchronization attributes.

Group Description Examples
Omit Libraries to omit from replication. System libraries, temporary work libraries, QGPL.
Don't Journal Libraries to replicate without journaling. Libraries containing few or no database files. Libraries containing only source files.  Libraries that rarely change.
Journal Libraries to replicate with journaling. Libraries containing database files, data areas or data queues which are updated regularly.

Tip: Use the printout from the Display Library Size (DSPLIBRSF) command generated in step 2 above to help you categorize your libraries.  For each library, the report the shows the total number of objects, the number of objects that could be journaled and the number of objects that should be journaled.

Objects that could be journaled include non-source database files, data areas and data queues.  Objects that should be journaled include any journalable object which is updated regularly.  (The report looks for objects updated within the last 7 days.)

8. Set Synchronization Attributes

Synchronization attributes can be set for each library, IFS directory and system object type (*USRPRF, *SYSVAL, etc) to tailor how RSF synchronizes these to the target machine.

For libraries you can specify:

For IFS directories you can specify:

For system information you can specify:

If you do not set synchronization attributes for a particular library, IFS directory or system object type, default values will be used.

Libraries: You set library synchronization attributes by running the Change Library Sync Attributes (CHGRSFSA) command for each library, or by creating a list and running the Set Sync Attributes For a List (SETRSFSA) command to change the attributes for all libraries in the list.  See the command help text for more information.

IFS directories: You set synchronization attributes for IFS directories by running the Change IFS Sync Attributes (CHGRSFISA) for each directory to be synchronized.  See the command help text for more information.

System info: You can set synchronization attributes for system information by running the Change System Sync Attributes (CHGRSFSSA) command for each system object type.

In addition, synchronization attributes can be set from the Work With Sync Attributes (WRKRSFSA) display.

9. Create Synchronization Start Programs

Starting replication for one or two libraries is easy.  Starting replication for dozens or hundreds of libraries and IFS directories can be complicated.  How do you make sure that you specify all of the start options you want in the exact same way each time?  Use a Synchronization Start Program.

Synchronization Start Programs incorporate RSF synchronization commands as needed to start synchronization for your environment.

To begin synchronizing libraries, you use the Synchronize Libraries (SYNCLIBRSF) command or the Synchronize List of Items (SYNCLSTRSF) command or the Synchronize From Attributes (SYNCATTRSF) command.

To begin synchronizing IFS directories, you use the Synchronize IFS Directories (SYNCIFSRSF) command or the Synchronize From Attributes (SYNCATTRSF) command.

To begin synchronizing system information, you use the Synchronize System Info (SYNCSYSRSF) command or the Synchronize List of Items (SYNCLSTRSF) command.

Two model Synchronization Start Programs are included in RSFTOOLS.  See source members STRSYNCPRD and STRSYNCBKP in file RSFTOOLS/QCLSRC.  To create your own programs, copy the model programs to library RSFUSER and modify them there.  (Do not store your programs in libraries RSF or RSFTOOLS or they may be lost the next time you upgrade RSF.)  For complete system integrity, you should replicate the library containing the source and object for your Synchronization Start Programs to the target machine.

We recommend creating a new library called RSFUSER in which to store source and objects for user-written role swap and synchronization start programs.  See step 2 above for information about creating library RSFUSER.

Note that your Synchronization Start Program will be passed two parameters, as shown in the source for the model programs:

Once you've created your program, tell RSF where to find it by using the Change RSF Defaults (CHGRSFDFT) command.  Prompt the command, page down almost to the end and set the "Sync start program" (SYNCSTR) parameter.

After creating your Synchronization Start Program and setting the RSF defaults, you can use the Start Synchronization (STRSYNCRSF) command (or RSFHA menu option 20) to consistently and easily start replication for your environment.  The STRSYNCRSF command does some initial processing and then passes control to your program.  The STRSYNCRSF command prompt is shown below.

                   Start Synchronization (STRSYNCRSF)

Type choices, press Enter.

User program to call . . . . . . my_pgm        Name, *RSFDFT
  Library  . . . . . . . . . . .   RSFUSER     Name, *LIBL, *CURLIB
Submit sync jobs . . . . . . . . *YES          *YES, *NO

                      Additional Parameters

Full save:
  Option . . . . . . . . . . . . *AUTO         *AUTO, *YES, *MANUAL
  Manual type  . . . . . . . . .               *RESET, *BEGIN, *END
Check connection to server . . . *NONE         Name, *NONE
Times to try connection . . . .  1             Number, *NOMAX
Minutes between tries . . . . .  10            Number  
 

Note: To be prepared for a role swap, you must create different synchronization start programs on the production and backup machines.

10. Start Synchronization

You are now ready to begin synchronization.  See Initial Library Copy  in the Things to Consider section below to understand the different options for establishing the first synchronization boundary.

If you created a Synchronization Start Program, you can use the Start Synchronization (STRSYNCRSF) command to begin.  Otherwise, use the following commands to begin:

Libraries:    Synchronize Libraries (SYNCLIBRSF)
IFS Directories:    Synchronize IFS Directories (SYNCIFSRSF)
System Information::    Synchronize System Info (SYNCSYSRSF)

For any of the above commands, specify a job description and a non-zero value (or *ATTR) for the "Repeat Every" (REPEAT) parameter to synchronize incremental changes at regular intervals.

Note: You can also start synchronization jobs from the Work With Sync Attributes (WRKRSFSA) display.

11. Modify Your System Start Program

To ensure that replication is started automatically whenever you IPL your system, you should add some instructions to your system start program.  The system start program is run by the operating system automatically when your controlling subsystem is started at the end of the IPL process.

You can use the following command to determine the name and library of your system start program:

DSPSYSVAL SYSVAL(QSTRUPPGM)

See source member STRUPPGM in file QCLSRC in RSFTOOLS for the recommended RSF instructions to add to your system start program. 

Simply copy the code from the STRUPPGM model in RSFTOOLS  to the source for your system start program.  Then, recompile your program into the library indicated in your QSTRUPPGM system value.

12. Create a System Monitor Program

With replication up and running, you should check periodically to be sure things are operating smoothly.  Better yet, have RSF monitor your system for you! 

With a System Monitor Program,  RSF checks critical system functions and sends an email or text message if anything needs attention.  Among other things, you can have RSF check for:

You can also define your own conditions and have RSF monitor those as well.

Source for model System Monitor Program is provided in source member CHKCDN in RSFTOOLS/QCLSRC.  Create your own program by copying the model to QCLSRC in library RSFUSER and modifying it there.

When your program is ready, use the system job scheduler to run your monitor program two or more times per day. (See below.)

13. Add Job Schedule Entries

Automate various RSF housekeeping and cleanup tasks by adding entries to your system job scheduler.  Program ADDRSFSCDE in RSFTOOLS makes this easy.

Source for ADDRSFSCDE is provided in RSFTOOLS/QCLSRC.  Create your own program by copying the model to QCLSRC in library RSFUSER and modifying it there.

Once you've adjusted program ADDRSFSCDE to suit your needs, just run it to add all of the recommended job schedule entries.


Things to Consider

 

Initial Library Copy

When you first synchronize a library or IFS directory, RSF will copy the entire library or directory to the target to ensure the versions are identical.  All  source and target objects must begin with  identical ownership and attributes, as well as identical data.

If you know the libraries are identical or you would like to copy the library manually, use the FULLSAVE(*MANUAL) option the first time you synchronize the library with the SYNCLIBRSF command.  This tells RSF to skip the initial copy.  Only changes made after this point are synchronized.  You will need to press F10 to see the FULLSAVE parameter. 

Please note:   If journaling is used for replication and you manually create the initial save image that will be used to restore an identical copy to the target machine, you must run the SYNCLIBRSF command with the FULLSAVE(*MANUAL) option twice: once before saving the library with SAVACT(*LIB), and again after the save-while-active checkpoint has been reached (local journaling) or after the library has been restored onto the target (remote journaling.) 

The first manual sync point ensures that journaling is properly initialized for all mirrored objects.  You should specify FULLSAVE(*MANUAL *BEGIN) for the first manual sync point.

The second manual sync point tells RSF to begin synchronizing changes at the appropriate point.  You should specify FULLSAVE(*MANUAL *END) for the second manual sync point.

In all cases, do not begin the actual synchronization, using the default FULLSAVE(*AUTO) option, until the initial copy of the library has been restored to the target.

See the online help text for the FULLSAVE parameter on the Synchronize Libraries (SYNCLIBRSF) command for more information.

 

Logicals and Physicals in Different Libraries

If you store logical files in a different library from the physical files they're based on, you have a cross-library dependency.  This is not the recommend way to organize your data because it is more complicated to manage.

Tip: Use the Display Cross-Lib Dependencies (DSPDBRRSF) command in library RSFTOOLS to check for cross-library dependencies.

In most cases, you can replicate your libraries in any order and RSF will manage any cross-library dependencies automatically.

Note that when a library is synchronized for the first time, RSF sends the entire library to establish a clean synchronization boundary.  In the process, RSF will want to clear the library on the target machine before restoring the copy saved from production.  If any objects cannot be cleared, inquiry message RSF3136 is sent to the system operators message queue on the target machine.  The text for message RSF3136 is

Message . . . . : Unable to clear library &1. (C R I)

Cause . . . . . : An RSF synchronization job needs to clear library &1 on this machine before it can continue but one or more objects in the library can not be deleted.

Recovery . . . : Do one of the following:

-- Type C to cancel the synchronization job.

-- Determine which objects in &1 can not be deleted, correct the problem and then type R to retry the operation and let the synchronization job continue.

-- Type I to ignore the clear operation and let the synchronization job try to continue without clearing the library.

To resolve the problem, manually delete any remaining objects in the target library and then answer the message with an 'R', allowing the replication of the library to continue.

 

Initial Journaled Object Copy

When a value other than *NONE is specified for Journal on the Change Library Sync Attributes (CHGRSFSA) command, physical files, data areas and data queues will be journaled.  You can start journaling yourself but we recommend that you let RSF start journaling for the appropriate objects automatically.

If RSF starts journaling for an object, it will automatically send the entire object to the target to ensure that both copies are in sync.  If you know the objects are identical and you would like to skip this step, use the FULLSAVE(*MANUAL) option as described above.  If you choose to start journaling yourself before beginning library synchronization, be sure to use the correct journal and to capture before and after images if updating by key is desired.

 

When to Journal

When replicating a library that contains large database files or one which will be in use on the target machine, we recommend synchronizing at the record level by specifying a value other than *NONE for Journal on the Change Library Sync Attributes (CHGRSFSA) command.  If two-way synchronization is planned, we recommend specifying *KEY for the "Apply journal entries by" (APPLY) parameter as well.

When replicating an IFS directory that contains very large files or files which may be locked and unable to be saved on a regular basis, we recommend synchronizing at the byte level by specifying a value other than *NONE for Journal on the Change IFS Sync Attributes (CHGRSFISA) command.

 

When not to Journal

When replicating a library containing source files, journaling is not desirable because the source members are updated wholesale by source editors.  For these libraries, we recommend specifying *NONE for Journal on the Change Library Sync Attributes (CHGRSFSA) command.

You can specify a synchronization journal for any library and the library will be synchronized properly.  However, it is more efficient if you specify *NONE for Journal on the Change Library Sync Attributes (CHGRSFSA) command when

 

Remote Journaling

In general, you can use remote journaling any time journaling is used to replicate a library or IFS directory.

Advantages of remote journaling:

Disadvantages of remote journaling:

Note: Remote journaling uses TCP ports 446 (DRDA),  and 447 (DDM) and 3777 (remote journaling proper).  If you intend to use remote journaling and your source and target machines are separated by a firewall, be sure to open ports 446, 447 and 3777.

In addition, you should use the Change DDM TCP/IP Attributes (CHGDDMTCPA) command on the target machine to change the "Password Required" option to *NO, as other options do not work in most environments.

 

Triggers and Library List Considerations

When triggers are associated with your database files, these triggers may be invoked on the target as well as on the source system.  You must ensure that the trigger programs exist on the target.  In addition, the RSF job that applies journal changes on the target must be able to find any trigger programs.  If necessary, add libraries containing trigger programs to the Initial Library List parameter of job descriptions RSF/RSFTCP2 and RSF/RSF.  You can use the Print Trigger Programs (PRTTRGPGM) command on the source machine to help identify triggers.

Please note: If your trigger programs update files that are being replicated by RSF, there is no need to run these programs on the target machine.  In this case, we recommend suspending your trigger programs on the target machine with the Change PF Trigger (CHGPFTRG) command, specifying *DISABLED for "Trigger state."  The command Change PF Triggers (CHGTRGRSF) is included with RSF to allow you to easily change the trigger state of many physical files at once.  In addition, you should specify *YES for the "Disable triggers on target" (DISTRG) parameter on the Change Library Sync Attributes (CHGRSFSA) command for any library that contains files with triggers that should be disabled on the target machine.

 

Multiple Journals Per Library

It's simpler if all database files, data areas and data queues in a given library are journaled to the same journal.  However, if you have a need to use multiple journals for a single library, RSF can still replicate all of the changes.  To accomplish this

System 36 Environment

If you are running the System 36 environment on your systems:

 


Is Everything OK?

How can you tell if the synchronization process is doing its job, without errors?

The best way is to use a System Monitor Program to notify you by email or text message if there's a problem.  See here for more info.  You can also do the following:

On the production (source) machine:

On the backup (target) machine:

Remember:  RSF repairs most synchronization errors automatically, within a cycle or two.  If you see an error that appears to go away after a few cycles, it has most likely been fixed already by RSF.

 


Role Swap

Overview

At any given time in a typical two-machine High Availability environment, one machine acts as the source or production machine and the other acts as the target or backup machine.  Users interact only with the production machine.  Replication keeps the backup machine synchronized with the production machine; changes to the production machine are automatically mirrored on the backup machine.

The machine that usually plays the production role is called the primary machine.  The machine that usually plays the backup role is called the secondary machine.

A role swap switches the perspective and the flow of data.  If before the role swap, A was the production machine and B was the backup  machine, then after the role swap B acts as the production machine and A (when available) acts as the backup machine.  Ideally, users experience minimal disruption while the swap is occurring.  Once the role swap is complete, users should not be able to detect the difference.

A role swap can be planned or unplanned

A planned or deliberate role swap might occur for many reasons, including:

An unplanned role swap occurs when a hardware or other error on the production machine forces that  machine out of service.  In such a case, the backup machine is swapped into production and the former production machine will act as the backup machine and target for replication once the error is corrected and  it again becomes available for use.

 

Preparation

There are several steps you must take in order to be prepared for a role swap.  The more thorough your planning and advance preparation, the smoother your role swaps will be.

  1. Ensure that all needed system information,  libraries and IFS directories are being replicated.  After having performed a role swap is the wrong time to discover that a needed library was not being replicated.  This is the most important item in the list.
     
  2. Synchronize configuration information from production to backup and from backup to production.  When synchronizing from production to backup, we recommend synchronizing to target library name RSFCFGPRD.  When synchronizing from backup to production, we recommend synchronizing to target library name RSFCFGBKP.

    The Change System Sync Attributes (CHGRSFSSA) command is used to define the target library to use when synchronizing *CFG information.  The Synchronize System Info (SYNCSYSRSF) command is used to perform the actual synchronization.

    We recommend synchronizing configuration information at least daily, in both directions.
     
  3. Ensure that library RSF is synchronized in both directions as follows:
     
    • Do not journal the objects in RSF (specify *NONE for the Journal parameter.)  Otherwise, it will be very difficult to upgrade to a future release of RSF.
       
    • Synchronize to a different target library name.  (Important!)
       
    • Synchronize library RSF from production to backup and from backup to production.  When synchronizing from production to backup, we recommend synchronizing to target library name RSFPRD.  When synchronizing from backup to production, we recommend synchronizing to target library name RSFBKP.
       
    • We recommend synchronizing library RSF every hour or two, in both directions.
       
    • You can use the Change Library Sync Attributes (CHGRSFSA) to omit all RSF *PGM objects from synchronization.  Only non-program RSF objects need to be synchronized.
       
  4. Create Synchronization Start Programs on both the production and backup machines so that your replication environment can be started consistently.  See above for more details.  The Synchronization Start Program on the backup machine only needs to start synchronization for configuration information and for library RSF, as described in points 2 and 3 above.
     
  5. Create a Production to Backup role swap program.  See below for more information.
     
  6. Create a Backup to Production role swap program.  See below for more information.
     
  7. Use the Change RSF Defaults (CHGRSFDFT) command (option 80 on menu RSFHA) to set the value for the Current Replication Role (ROLE) parameter.

Create a Production-to-Backup Role Swap Program

A Production to Backup Role Swap Program contains all the steps needed to convert the production machine to the backup role.  Creating your own program allows you to customize the role swap process.

Two model programs are included in RSFTOOLS.  See source members SWAPTOBKP and SWAPTOBKP2 in file RSFTOOLS/QCLSRC.  To create your own program, copy one of the two programs to your own library and modify it there.  Do not store your program in libraries RSF or RSFTOOLS or it may be lost the next time you upgrade RSF.  For complete system integrity, you should replicate the library containing  your program to the target machine.

We recommend creating a new library called RSFUSER in which to store source and objects for user-written role swap and synchronization start programs.

The difference between SWAPTOBKP and SWAPTOBKP2 is that the former assumes IP addresses and system configuration will be swapped.  The latter does not.

Note that your Production to Backup Role Swap Program will be passed no parameters.

Once you've created your program, tell RSF where to find it by using the Change RSF Defaults (CHGRSFDFT) command.  Prompt the command, page down almost to the end and set the "Role swap to backup" (SWAPTOBKP) parameter.

Create a Backup-to-Production Role Swap Program

A Backup to Production Role Swap Program contains all the steps needed to convert the backup machine to the production role.  Creating your own program allows you to customize the role swap process.

Two model programs are included in RSFTOOLS.  See source members SWAPTOPRD and SWAPTOPRD2 in file RSFTOOLS/QCLSRC.  To create your own program, copy SWAPTOPRD to your own library and modify it there.  Do not store your program in libraries RSF or RSFTOOLS or it may be lost the next time you upgrade RSF.  For complete system integrity, you should replicate the library containing your program to the target machine.

We recommend creating a new library called RSFUSER in which to store source and objects for user-written role swap and synchronization start programs.

The difference between SWAPTOPRD and SWAPTOPRD2 is that the former assumes IP addresses and system configuration will be swapped.  The latter does not.

Note that your Backup to Production Role Swap Program will be passed no parameters:

Once you've created your program, tell RSF where to find it by using the Change RSF Defaults (CHGRSFDFT) command.  Prompt the command, page down almost to the end and set the "Role swap to production" (SWAPTOPRD) parameter.

 

Performing an Unplanned Role Swap

In the event that the primary machine unexpectedly goes down, swap the secondary machine into production:

  1. On the secondary machine, select option 89 from the RSFHAT menu.   You must be signed on as QSECOFR or equivalent.  You you must also run the option from the system console or specify *YES for "Run in batch.".  

     
                         Role Swap to Production (SWAPPRDRSF)

    Type choices, press Enter.

    User program to call . . . . . . *RSFDFT          Name, *RSFDFT
      Library  . . . . . . . . . . .                  Name, *LIBL, *CURLIB
    Run in batch . . . . . . . . . . *NO              *YES, *NO
    Are you sure?  . . . . . . . . . *NO              *YES, *NO
    Current role . . . . . . . . . . *PROD


    Fill in the command prompts and press Enter to begin the role swap.  Status messages keep you informed of progress.  When the process completes, users can resume their work.  The former backup machine is now acting as the production machine.
     

     

  2. Once the problem with the primary machine is resolved and the machine becomes available, convert it to the backup role by running option 89 from the RSFHA menu.  Again, you must be signed on as QSECOFR or equivalent and you must either run the option from the system console or specify *YES for "Run in batch.".  

     
                         Role Swap to Backup (SWAPBKPRSF)

    Type choices, press Enter.

    User program to call . . . . . . *RSFDFT          Name, *RSFDFT
      Library  . . . . . . . . . . .                  Name, *LIBL, *CURLIB
    Run in batch . . . . . . . . . . *NO              *YES, *NO
    Are you sure?  . . . . . . . . . *NO              *YES, *NO
    Current role . . . . . . . . . . *PROD

    When processing completes, the machine is ready to receive changes from the production (secondary) machine.

 

Performing a Planned Role Swap

A planned role swap can take several forms.  The following table summarizes the options.

Type Description Relative Benefits
Full Complete swap of both machines Takes the longest, but exercises role swaps in both directions on both machines.  Any user data changes made during the role swap are preserved.
Archive Primary machine is taken off line but is not swapped.  Only the secondary machine is swapped.  RSF's archive/refresh tools are used to roll back any changes made on the secondary machine while testing the role swap. Faster than Full, but slower than Compare.  Good simulation of an unplanned role swap.  User data changes made during the role swap are discarded.

Before swapping the secondary machine to the production role, all libraries and directories are archived.  After returning the secondary machine to the backup role, all libraries and directories are refreshed from the archive.

Compare Primary machine is taken off line but is not swapped.  Only the secondary machine is swapped.  RSF's compare tools are used to roll back any changes made on the secondary machine while testing the role swap. Faster than Archive but slower than Read Only.  Good simulation of an unplanned role swap.  User data changes made during the role swap are discarded. 

After returning the secondary machine to the backup role, a compare of all libraries and directories is launched from the primary machine.  Objects that were changed on the secondary machine are refreshed when replication from the primary to the secondary machine resumes.

Read Only Primary machine is taken off line but is not swapped.  Only the secondary machine is swapped. Fastest option.  Users can view but not change any data during the role swap. 

(RSF does not enforce the read-only restriction during the role swap.  The system administrator is responsible for enforcement.)

To perform a Full role swap:

  1. Warn users to sign off the primary (production) machine.
     
  2. Swap the primary machine to the backup role by running option 89 from the RSFHA menu.   You must be signed on as QSECOFR or equivalent and you must either run the option from the system console or specify *YES for "Run in batch.".  Wait for the swap to complete.
  1. Swap the secondary  machine to the production role by running option 89 from the RSFHAT menu.   When the swap completes, users can sign back on and resume work.  The secondary machine is now the production machine.
     
  2. When you're ready to swap back, warn users to sign off the secondary machine.
     
  3. Swap the secondary machine to the backup role by running option 89 from the RSFHA menu.   Wait for the swap to complete.
     
  4. Swap the primary  machine to the production role by running option 89 from the RSFHAT menu.   When the swap completes, users can sign back on and resume work.  The primary machine is again the production machine.

 

To perform an Archive role swap:

  1. Warn users to sign off the primary machine.
     
  2. Take the primary machine offline by doing one of the following, depending on the requirements of your environment:
     
    • End subsystem QINTER, end the RSF server function and hold the synchronization job queue RSFUSER/RSFHA.
    • End all subsystems
    • Power down the system.
  1. Swap the secondary  machine to the production role by running option 89 from the RSFHAT menu.   The secondary machine is now the production machine, but the next step should be completed before any changes are made to user libraries.
     
  2. Use the Archive from Sync Attributes (ARCATTRSF) command to archive all production libraries and IFS directories on the secondary machine.  When the the archive is complete, users can sign on and make changes.
     
  3. When you're ready to swap back, warn users to sign off the secondary machine.
     
  4. Use the Refresh from Sync Attributes (REFATTRSF) command to refresh all production libraries and IFS directories on the secondary machine.  Note: Be sure to omit library RSF from the refresh.
     
  5. Swap the secondary machine to the backup role by running option 89 from the RSFHA menu.   Wait for the swap to complete.
     
  6. Restart the primary machine.  When the machine comes back up, users can sign back on and resume work.  The primary machine is again the production machine.

 

To perform a Compare role swap:

  1. Warn users to sign off the primary machine.
     
  2. Take the primary machine offline by doing one of the following, depending on the requirements of your environment:
     
    • End subsystem QINTER, end the RSF server function and hold the synchronization job queue RSFUSER/RSFHA.
    • End all subsystems
    • Power down the system.
  1. Swap the secondary  machine to the production role by running option 89 from the RSFHAT menu.   When the swap completes, users can sign on and make changes.  The secondary machine is now the production machine.
     
  2. When you're ready to swap back, warn users to sign off the secondary machine.
     
  3. Swap the secondary machine to the backup role by running option 89 from the RSFHA menu.   Wait for the swap to complete.
     
  4. Restart the primary machine, but don't allow users to update any user libraries until the next step is completed.
     
  5. Use the Check Defined Items (CHKATTRSF) command on the primary machine to identify changed objects on the secondary machine.  Be sure to press F10 to see all parameters and specify REFRESH(*YES).  When all compare jobs complete, users can sign back on and resume work.  The primary machine is again the production machine.  Any changed objects on the secondary machine will be refreshed in the normal course of replication.
     

To perform a Read Only role swap:

  1. Warn users to sign off the primary machine.
     
  2. Take the primary machine offline by doing one of the following, depending on the requirements of your environment:
     
    • End subsystem QINTER, end the RSF server function and hold the synchronization job queue RSFUSER/RSFHA.
    • End all subsystems
    • Power down the system.
  1. Swap the secondary  machine to the production role by running option 89 from the RSFHAT menu.   When the swap completes, users can sign on view items without making any changes.  The secondary machine is now the production machine.
     
  2. When you're ready to swap back, warn users to sign off the secondary machine.
     
  3. Swap the secondary machine to the backup role by running option 89 from the RSFHA menu.   Wait for the swap to complete.
     
  4. Restart the primary machine.  When the machine comes up, users can sign back on and resume work.  The primary machine is again the production machine.
     

 


Special Considerations for Two-Way Mirroring

Two-way mirroring is needed if the objects being synchronized can be updated on both the source and target machines.  This type of replication can present special challenges.

A more typical application mix on a single machine would allow different users and different applications to be interacting with and updating a single data set.  In this scenario, integrity is maintained by the operating system via record and object locks.

Even a distributed application which accesses a single data set via DDM or SQL gets a lot of help from the operating system in maintaining integrity.

With replication or mirroring, however, there are two distinct copies of the data.  When both copies can be updated directly by user applications, extra care must be taken to ensure that the copies do not diverge.  RSF helps facilitate two-way mirroring in the following ways:

User Exit Programs

Some file updates require special handling in a two-way mirroring environment.  Here's a classic example:

Assume a given file contains a counter or total field called NUMBER. NUMBER could represent inventory on hand, last used order number, last date accessed, etc. With one-way replication, changes to the file on machine A can be applied literally to machine B. With two-way mirroring, however, an increment to NUMBER in a record on machine A should result in an equivalent increment to NUMBER in the same record on machine B, keeping in mind that NUMBER may have been incremented by a program local to machine B before the change from A is received. This can easily be handled by an exit program which looks at the before and after values for NUMBER and increments the field in the target record appropriately.

Exit programs are passed the following parameters:

It is the user's responsibility to ensure that exit programs are well behaved.  In particular, they should:

See source member UPDATEEXIT in RSFTOOLS/QCSRC for a sample exit program.

To associate an exit program with a target file, use the Change Unique Key Association  (CHGRSFUKEY) command on the target machine.  Specify the physical file name for the PFILE parameter and the qualified exit program name for the EXITUPD parameter.

Do not place your exit programs in libraries RSF or RSFTOOLS as they may be lost at the next RSF upgrade.

 

 


Change PF While Active

Use this function to change the layout of a physical file with minimal disruption. The file can be in use during the change except for a brief time at the end when the new format is installed.

For large files, using the system CHGPF or the ALTER TABLE functions to change a file layout is not always practical because these operations run for a long time during which the file is unavailable.

The Change PF While Active (CHGPFRSF) command makes changing large file layouts practical. The file format is changed in the background and all  record and member changes made during the update process are preserved.

Note:

  1. As always when changing a file layout, you must coordinate the installation of this change with updated programs that reference the new file format.
     

  2. During the change process, a copy of the original file is made in a temporary library. You must ensure that adequate disk space is available in the auxiliary storage pool (ASP) of the original file to allow for the temporary copy.

The prompted version of the CHGPFRSF command is shown below. Click on the image to jump to a particular command parameter description.

                          Change PF While Active (CHGPFRSF)

Type choices, press Enter.

Physical file . . . . . . . . . Name           
  Library . . . . . . . . . . .   *LIBL          Name, *LIBL, *CURLIB
Action to perform . . . . . . . *PREP            *PREP, *SYNC, *ENDSYNC...
Restart . . . . . . . . . . . . *NO              *YES, *NO
Ignore inquiry messages . . . . *NO              *YES, *NO
Ignore journal apply errors . . *NO              *YES, *NO
Source file . . . . . . . . . . Name                   
  Library . . . . . . . . . . .   *LIBL          Name, *LIBL, *CURLIB
Source member . . . . . . . . . *FILE            Name, *FILE
Source type . . . . . . . . . . *AUTO            *AUTO, *DDS, *SQL
Message queue . . . . . . . . . Name             
  Library . . . . . . . . . . .   *LIBL          Name, *LIBL 

The parameters for the CHGPFRSF command are described below in the order that they appear on the command prompt.

Physical File

Enter the qualified name of a file to change.

The possible file values are:

name: Enter the name of an existing physical file.

The possible library values are:

*LIBL: The file is found in the job library list.
*CURLIB The file is found in the current library.
name: Enter the name of the library that contains the file to be changed.

Action to Perform

Specify the action to perform.

Note: The file must be prepared before it can be synchronized or installed.

The possible values are:

*PREP: The file is prepared. A temporary work library is created, a copy of the file is placed in the work library and the record layout of the copy is changed. This operation may run for a long time. The original file is fully accessible during this step.
*SYNC A batch job is launched which keeps the file copy up to date with the original.

Note: The *SYNC step is launched automatically upon successful completion of the *PREP step. Use the *SYNC option to restart synchronization if it was ended manually or the system was powered down before the changed file could be installed. When restarted, synchronization automatically picks up where is left off.

The original file is fully accessible during this step.
*ENDSYNC: Use this option to end synchronization between the original file and the copy, if desired.
*INSTALL Use this option to install the updated file into production. The file cannot be used during the install step. Any outstanding changes to the file are  applied before it is installed.

To minimize the time during which the file is inaccessible, wait until most record and member level changes have been applied to the file copy before beginning the installation. You can use the Message Queue  (MSGQ) parameter to be notified when this occurs.
*CANCEL Use this option to cancel the operation for a file that has been prepared but not installed.

Restart

Specify whether to restart the *PREP operation.

This parameter is ignored unless *PREP is specified for "Action to Perform".

The possible values are:

*NO: If preparation of the file has started and the install step has not completed, the operation will end in error.
*YES: The *PREP operation is restarted for the file, even if the file was already prepared and ready to be installed.

Ignore Inquiry Messages

Specify whether to ignore certain inquiry messages during the *PREP and *INSTALL steps. Specify *YES for this parameter if you want to ensure that the operation runs unattended.

The possible values are:

*NO: Any inquiry messages must be answered explicitly.
*YES: Inquiry messages CPA7027 and CPA32B2 are ignored. A default response of "I" is assumed.

Ignore Journal Apply Errors

Specify whether to install the changed file even if some of the record-level changes made by users while the file was being prepared could not be replicated successfully.

This parameter is ignored unless *INSTALL is specified for "Action to Perform".

Note: You can use a command like "DSPLOG MSGID(RSF4211)" to obtain more information about any journal apply errors.

The possible values are:

*NO: If any record-level changes could not be replicated to the work file, the work file cannot be installed in production.
*YES: The updated file can be installed even if some record-level changes made during file preparation could not be replicated to the work file.

Source File

Specify the source file that contains DDS or SQL instructions for changing the file layout.

This parameter is required if *PREP is specified for "Action to Perform". Otherwise, it is ignored.

The possible file values are:

name: Enter the name of an existing source file.

The possible library values are:

*LIBL: The source file is found in the job library list.
*CURLIB The source file is found in the current library.
name: Enter the name of the library that contains the source file to use.

Source Member

Specify the source member that contains DDS or SQL instructions for changing the file layout.

This parameter is ignored unless *PREP is specified for "Action to Perform".

The possible values are:

*FILE: The member name is the same as the name of the file being updated.
name: Enter the name of the member to use.

Source Type

Indicate whether the source member contains DDS or SQL specifications.

The possible values are:

*AUTO: The source type of the source member is used to determine the type of specifications contained in the member. If the source type begins with "PF", *DDS is assumed. Otherwise, *SQL is assumed.
*DDS: The source member contains DDS specifications suitable for use with the CRTPF or CHGPF command.
*SQL: The source member contains SQL statements, including at least one "ALTER TABLE" statement.

Note: The name of the file specified on ALTER TABLE statements in the source member is ignored. RSF replaces the file name in ALTER TABLE statements with w/n, where w is the name of the work library for this operation, and n it the name of the file to be changed.

Message Queue

If specified, this is the queue to which massages are sent indicating the progress of the change operation. You can monitor this message queue in another job to determine when the *PREP, *SYNC and *INSTALL steps have completed.

The messages and their meanings are as follows:

RSF7270: Sent when the *PREP step completes.
RSF7271: Sent the first time all record-level changes to the file have been processed. This includes all changes made since the beginning of the *PREP step.
RSF7272: Sent when installation of the updated file is complete.

The possible message queue values are:

name: Enter the name of an existing message queue.

The possible library values are:

*LIBL: The message queue is found in the job library list.
*CURLIB The message queue is found in the current library.
name: Enter the name of the library that contains the message queue.