Here are some of my tips for using the Platform Suite for SAS when you have Development, Test and Production environments to support your SDLC.
Install and configure one instance of Platform LSF and Platform Process Manager
If you have separate machines for your development, test and production environments then this tip won’t have much value for you.
SAS manages to nicely separate itself into development, test and production environments on the same machine by using different port numbers, and these port numbers usually end in the same number as the Level it is associated with.
| Environment | Level | Metadata Server Port |
| Development | Lev3 | 8563 |
| Test | Lev2 | 8562 |
| Production | Lev1 | 8561 |
The Platform Suite for SAS, which consists at it’s core of Platform LSF and Platform Process Manager does not lend itself to the same, nice, easy separation. While it is possible to install and configure multiple instances of the Platform software on the same machine to use different ports for different environments, in my view this is overly complicated and unnecessary. I find that a more productive approach is to have a single instance and achieve environment separation through the use of “Batch User Id’s”. In this configuration, each SAS environment is configured to use a different LSF Batch user and although there is only a single Platform Suite instance, the separation of environments is maintained. An example view of this using the Platform Flow Manager is shown below:
Using the Platform Flow Manager, there will be one Tree of Job Flows for each environment (user).
Allowing Developers to Schedule Flows
Part of SAS ETL development should also be the creation and testing of the associated jobs schedules. The common way I have seen this previously done is either the Developer logs into SAS Management Console (SMC) as the SAS Administrator, or the Admin/Lead Developer logs into the SMC as the SAS Administrator so they can create and deploy Job Flows using the Schedule Manager plug-in. In some cases I have seen customers with a “Scheduling Admin” user in SMC which has the “lsfadmin” account associated with it. I think that these scenarios are usually a result of not fully understanding the relationship between the SAS products (SMC and DI Studio) and the Platform products (LSF and Process Manager), security/account requirements of each and how SAS and Platform interact with each other. It is easier to “just get it to work” which usually results in using an account with more power than is actually required.
Disclaimer: The steps I describe below is one method of many to achieve the desired result. If you are unsure of anything please consult a SAS specialist – do not make any changes to the SAS Metadata or Configuration unless you understand why you are making those changes and have a working backup.
Preparing the Platform Process Manager
One of the first things to check is the Authentication Domain of the Platform Process Manager in SMC. Log into SMC as the SAS Administrator and locate the Platform Process Manager in the Server Manager plug-in. Right click on the Connection: Platform Process Manager object on the right and select Properties.
On the Options tab, check the Authentication Domain. If this specifies DefaultAuth or (None) then create a new Authentication Domain called LSFAuth by clicking the New… button. This will be the “Tag” used by SAS when trying to find a valid logon for the Platform Process Manager, and is the key for the “single signon” effect when giving users/developers the ability to schedule jobs from SMC.
Preparing a Scheduling Role
Previously access to the SMC was usually an “all or nothing” deal. The introduction of Roles within the SAS Metadata has made it easier to grant access to specific tasks within SMC. Now we can create a Role that will allow users to access the SMC and have access to the Schedule Manager. To create a new Role, log into SMC as the SAS Administrator or another user with the Metadata Server: User Administration Role or equivalent, and select New->Role from the User Manager plug-in.
Once this Role has been defined it can be given directly to individual Users or Groups. From an Administration perspective, it makes life easier if Roles are assigned to Group(s). Users come and go, but Groups usually stick around!
Create a Scheduling Group
Next create a Scheduling Users Group in SMC that will provide members access to the above Role, as well as provide access to the designated LSF Batch User Id.
On the Members tab, add each User or Group that you would like to give Scheduling capabilities to. In a Development environment this may be the SAS ETL Developers group or similar.
On the Groups and Roles tab, select the Management Console: Scheduling Role we defined previously.
On the Accounts tab, click the New button and then enter the LSF Batch User Id you want to use for this environment along with the password. This LSF Batch User Id must exist as a Host/Domain account – this is a normal requirement of LSF. As this will be used for “Outbound” authentication it is important that you enter the password. The Authentication Domain of the Account should match the Authentication Domain of the Platform Process Manager server in the Server Manager plug-in – LSFAuth if you are following along with these steps.
What a Scheduling User Experiences
I am not going to go through the core ETL development process, but we will assume that Job(s) have been created and Deployed from DI Studio with the next step being to create and schedule the Job Flow. The Scheduling process is done from SMC, so the developer/scheduler will log into SMC using their normal developer account. In this example I am logging in using IWA so that SAS automatically picks up my user id. Because I am a member of the SAS ETL Developers group, which is a member of the Scheduling Users group, which has the Management Console: Scheduling Role, I have access to the Schedule Manager plug-in in SMC.
I can now create a Job Flow and add my deployed DI Studio Job. In this example I have created a “TestFlow” Job Flow and added a “Test_Job” deployed job.
Choosing Schedule Flow… on my new Job Flow in SMC results in the following events happening behind the scenes:
- The associated Scheduling Server (Platform Process Manager) is checked in SAS Metadata to see what the Authentication Domain is. In this case the Authentication Domain is LSFAuth.
- SAS then checks who is currently logged in (Michael Dixon) and if their SAS Metadata Identity has an account for LSFAuth (No).
- As there is no LSFAuth account directly on the logged in Metadata Identity, the group memberships are checked until it finds the LSFAuth account on the Scheduling Users group.
- The user id and password for this LSFAuth account is then used by SAS to communicate with the scheduling server (this is why the password needs to be stored in Metadata).
Because of this there will be no password prompt and it should appear to “magically” work. One less username and password for people to remember!
If the above setup has not been done or a user that is not a member of the Scheduling Users group attempts to schedule a Job Flow then the following prompt, which you may already be familiar with, will appear:
Configuring the Platform Process Manager in this way allows for a more streamlined development process, with DI Studio developers able to schedule and test their own jobs under a standard LSF Batch User Id without compromising security.
Test and Production
Test and Production have the same configuration settings as above, except the LSFAuth account stored against the Scheduling Users group is changed appropriately. On our server we have the following Active Directory accounts for each environment:
| Environment | Level | Account |
| Development | Lev3 | SCORPIO\lsfdev |
| Test | Lev2 | SCORPIO\lsftest |
| Production | Lev1 | SCORPIO\lsfprod |
Since there should not be any development work happening in Test (no SAS ETL Developers group), another appropriate group or individual user(s) would be added as Members of the Scheduling Users group in Test.
In Production there may only be one user with the role of maintaining schedules added as a Member of the Scheduling Users group, or another method, such as creating an internal account may be utilised. There are many options/combinations that can be implemented based on individual site security requirements.
Michael,
ReplyDeleteThis blog is being bookmarked! Thank you for sharing your experience, I look forward to reading more,