Flat file account loading or user loading seems like a very basic feature of OIM, and something every admin should have a grasp on. Unfortunately, that seems not to be the case.
Though mostly used for 'Disconnected' app instances, a flat file account loading can also be executed for any connected application too, in case we're unable to do the recon with the OOTB recon schedulers.
the point here is, we're trying to write the account footprint in OIMdatabase, so things like Request data validator, Adapters,or pre-populate plug-ins do not come into play.
The key components of this set-ups are :
1. Reconciliation profile, created through design console from the resource object :
2. Data mapping in process definition :
3. Create Reconciliation Rule :
4. Loop-ups set-up :
in this lookup, define
5. Set up the input CSV file -
In the format - (we'll be using the delimiters defined in the above screenshot)
Copy and paste the input file directory name in scheduler, make sure there is only one csv file in that directory.
header1,header2,header3,header4
"ABC","AB BC CA","0","XY";"YZ";"ZX"
6. Set up the schema file generic_schema_APPNAME.properties file -
The schema file is used for reading the input.csv file, this is where you define the structure of the input file.
We'll just refer tot he sample schema file from Oracle so that we can get a grasp on every aspect of it, even though not all may be useful to us .
#List of fields
#FieldNames=UID,UserId,FirstName,LastName,email,Currency,Salary,status,JoiningDate,LastUpdated,Groups,Roles
FieldNames=header1,header2,header3,header4 (Mandatory)
#Unique ID Attribute (Mandatory)( This is the attribute that has the AccountId=true property in the foem designer)
UidAttribute=UID
#Account Name attribute (Mandatory)( This is the attribute that has the AccountName=true property in the foem designer)
NameAttribute=UserId
#Multivalued attributes
Groups.Multivalued=true
Roles.Multivalued=true
#Subfields for complex child form
Roles.Subfields=RoleName,Start_Date,End_Date
#Complex child form objectClass
Roles.EmbeddedObjectClass=MyROLES
#Datatypes (Default:String) (Optional)
Roles.Start_Date.DataType=Long
Roles.End_Date.DataType=Long
FirstName.DataType=String
JoiningDate.DataType=Long
#Incremental reconciliation attribute with datatype set to Long
LastUpdated.DataType=Long
#Parent and child form mandatory fields (Optional)
Roles.RoleName.Required=true
#Date format
SystemDateFormat=ddmmyy
#Account Status Attribute and Mapping
StatusAttribute=status
status.True=Enabled
status.False=Disabled
7. configure the IT resource -
Find the ITResource that you're going to use for this recon. It can be the general 'Flat File Accounts' ITResource , or the specific IT Resource for our application.
Change the value of the attribute 'Configuration Lookup' to 'Lookup.FlatFile.Configuration'.
Change the value of the attribute 'schemaFilep' to the schema file name with entire location , made in step 6 (ex : /u01/oracle/admin/shared/schemaFiles/generic_schema_APPNAME.properties).
(in case of cluster set up, make sure your schema file is in a shared location, or you're running the job from the managed server in which the schema file is kept.)
8. Configure the Scheduler "Flat File Accounts Loader" -
Run the scheduler.
9. Do a blank search on 'Event Management' .
Check whether the events are getting created with the key fields from step 2 or not, if not check the logs.
For each event generated , under RECONCILIATION DATA 'Attribute Name', 'Attribute Value', and 'OIM Mapped field'
Under "Matched User', we can check which user is mapped with the account.
Under Roles section, we'll see the multi-valued attributes.
Thanks for Reading !
NOTES :
If a delimiter containing more than one character (for example, $#) is specified in the flat file and in the main configuration lookup definition, then the following error is encountered:
Only single character delimiters are supported with the exception of "tab" and "space".
Configuring Fault Handling
Record level errors while parsing the file are logged in a separate file and will be saved in a directory named "failed" that the connector creates, within the flat file directory. The processed flat file will be saved in the following format:
FILENAME_dd-MM-yyyy_HH-mm-ss.EXT
In this format, FILENAME is the name of the flat file being archived. dd-MM-yyyy_HH-mm-ss is the date and time at which the connector started processing the file. EXT is the extension of the file.
For example, the filename will be saved in the following format:
acmeusers_29-08-2013_22-44-12.csv
Though mostly used for 'Disconnected' app instances, a flat file account loading can also be executed for any connected application too, in case we're unable to do the recon with the OOTB recon schedulers.
the point here is, we're trying to write the account footprint in OIMdatabase, so things like Request data validator, Adapters,or pre-populate plug-ins do not come into play.
The key components of this set-ups are :
1. Reconciliation profile, created through design console from the resource object :
- Search for / Create the 'Resource Object', in the design control, under 'Resource Management' option.
- Expand 'Reconciliation fields', and make sure to add all the attributes of the object form, with their type.
- Add child forms as 'Multi-valued Attributes '. Add each attribute of the child form under it.
- Mark as 'Required' only the attribute(s) which must be present in the object form.
- Click on 'Create Reconciliation Profile' button on the right hand upper corner --> This will RA_PROCESS_FORM_NAME table in the database.
2. Data mapping in process definition :
- Search for / Create the 'Process Definition' , in the design control, under 'Process Management' option.
- Expand the 'Reconciliation Field Mapping' part, click on 'Add Field Map' for adding new attributes mapping, click on 'Add Table Map' for adding new child forms mapping.
- The above mapping is between Resource Object and Process Form.
- Mark the fields as 'Key' which are unique for an user, and also present in the user profile in OIM.
3. Create Reconciliation Rule :
- Search for / Create the 'Reconciliation Rules', in the design control, under 'Development Tools' option.
- click on 'Add Rule Element' , and map an attribute from 'User Profile Data' , to 'Attribute'.
- Make sure you map the attribute(s) only which you've marked as 'Key' in step 2.
- We may use multiple 'Rules' based on multiple key attribute, but all of these rules can work together only with either AND or OR.
4. Loop-ups set-up :
- Open / Create 'Lookup.XYZ.UP.FF.Recon' --> This lookup will hold the mapping between the resource object fields and the 'FieldNames' in schema name(STEP : 6) or headers in the input CSV file (STEP : 5).
- Open 'Lookup.FlatFile.UM.Configuration' --> Put the lookup name (ex : Lookup.XYZ.UP.FF.Recon) in decode key against the code 'Recon Attribute Map'
in this lookup, define
- fieldDelimiter --> the character that distinguish one field from another
- subFieldDelimiter--> the character that distinguish subfields (i.e Start date,role code,role name sub fields under one field ROLES)
- multiValueDelimiter--> the character that distinguish multiple values of one field in one account
- textQualifier --> the character that defines a string as one testValue for one field, comes in use where you've space in the value.
- Put the user configuration lookup name (Lookup.FlatFile.UM.Configuration) in decode key against the code 'User Configuration Lookup'
5. Set up the input CSV file -
In the format - (we'll be using the delimiters defined in the above screenshot)
Copy and paste the input file directory name in scheduler, make sure there is only one csv file in that directory.
header1,header2,header3,header4
"ABC","AB BC CA","0","XY";"YZ";"ZX"
6. Set up the schema file generic_schema_APPNAME.properties file -
The schema file is used for reading the input.csv file, this is where you define the structure of the input file.
We'll just refer tot he sample schema file from Oracle so that we can get a grasp on every aspect of it, even though not all may be useful to us .
#List of fields
#FieldNames=UID,UserId,FirstName,LastName,email,Currency,Salary,status,JoiningDate,LastUpdated,Groups,Roles
FieldNames=header1,header2,header3,header4 (Mandatory)
#Unique ID Attribute (Mandatory)( This is the attribute that has the AccountId=true property in the foem designer)
UidAttribute=UID
#Account Name attribute (Mandatory)( This is the attribute that has the AccountName=true property in the foem designer)
NameAttribute=UserId
#Multivalued attributes
Groups.Multivalued=true
Roles.Multivalued=true
#Subfields for complex child form
Roles.Subfields=RoleName,Start_Date,End_Date
#Complex child form objectClass
Roles.EmbeddedObjectClass=MyROLES
#Datatypes (Default:String) (Optional)
Roles.Start_Date.DataType=Long
Roles.End_Date.DataType=Long
FirstName.DataType=String
JoiningDate.DataType=Long
#Incremental reconciliation attribute with datatype set to Long
LastUpdated.DataType=Long
#Parent and child form mandatory fields (Optional)
Roles.RoleName.Required=true
#Date format
SystemDateFormat=ddmmyy
#Account Status Attribute and Mapping
StatusAttribute=status
status.True=Enabled
status.False=Disabled
7. configure the IT resource -
Find the ITResource that you're going to use for this recon. It can be the general 'Flat File Accounts' ITResource , or the specific IT Resource for our application.
Change the value of the attribute 'Configuration Lookup' to 'Lookup.FlatFile.Configuration'.
Change the value of the attribute 'schemaFilep' to the schema file name with entire location , made in step 6 (ex : /u01/oracle/admin/shared/schemaFiles/generic_schema_APPNAME.properties).
(in case of cluster set up, make sure your schema file is in a shared location, or you're running the job from the managed server in which the schema file is kept.)
8. Configure the Scheduler "Flat File Accounts Loader" -
Run the scheduler.
9. Do a blank search on 'Event Management' .
Check whether the events are getting created with the key fields from step 2 or not, if not check the logs.
For each event generated , under RECONCILIATION DATA 'Attribute Name', 'Attribute Value', and 'OIM Mapped field'
Under "Matched User', we can check which user is mapped with the account.
Under Roles section, we'll see the multi-valued attributes.
Thanks for Reading !
NOTES :
If a delimiter containing more than one character (for example, $#) is specified in the flat file and in the main configuration lookup definition, then the following error is encountered:
Only single character delimiters are supported with the exception of "tab" and "space".
Configuring Fault Handling
Record level errors while parsing the file are logged in a separate file and will be saved in a directory named "failed" that the connector creates, within the flat file directory. The processed flat file will be saved in the following format:
FILENAME_dd-MM-yyyy_HH-mm-ss.EXT
In this format, FILENAME is the name of the flat file being archived. dd-MM-yyyy_HH-mm-ss is the date and time at which the connector started processing the file. EXT is the extension of the file.
For example, the filename will be saved in the following format:
acmeusers_29-08-2013_22-44-12.csv