Skip to main content

Loading Data and Reprocessing Documents

This chapter contains information about loading the transaction and balances data from your source system into SLT Replication Server.

Configuring SAP S/4HANA for Data Load

During the configuration, map the master data in Transaction with the data in insightsoftware using SAP MDG:

Important: insightsoftware recommends that you use MDG to map the data in Transaction with the data in SAP S/4HANA. Use MSC_Lookup_Data.xlsx to map the data if there are limitations in MDG, such as character length mismatch between the source data and the data in SAP S/4HANA.

  1. Configure the MDG business mappings in Central Finance/MDG for the following objects:
    • Company Code
    • General Ledger Account
    • Cost Center
    • Profit Center
    • Customer
    • Vendor
    • Material Number
    • Document Type
    • Tax Code
    • Withholding Tax Code

    For example, perform the following steps to configure the customer mapping:

    1. To upload customer mappings into SAP S/4HANA, log on to SAP S/4HANA, and type the transaction code FINS_CFIN_MAP_MANAGE.
    2. Perform the following steps in the Central Finance: Manage Mappings window:
      1. In Mapping Entity, select ERP customer number (ERP).
      2. Select a source system identifier.
      3. In the User Action area, select Upload Mappings.
      4. In the File Selection area, browse to the mapping file (.csv) location, and select the mapping to upload.
      5. Select Run. The customer mapping is uploaded to the system successfully.
  2. Create a master data file with the name MSC_Lookup_Data.xlsx to map the objects that are not mapped using MDG. Populate the lookup data in the file for objects, such as journal types, document type, tax code, withholding tax code, and logical system, using the sample format specified in the Sample_MSC_Lookup_Data.xlsx file at <SetupFolder>\<SourceSystem>\Sample_MSC_Lookup_Data.

    Note: Do not change the data in the Source and Lookup_Code columns.

  3. Define time periods for opening balances, such as customer, vendor, and general ledger account balances.
  4. Use the transaction code BD54 to create an external logical source system to represent your source system. The length of the logical source system name cannot be greater than 10 characters.
  5. If applicable, finalize the format as Universal file format to load data from a legacy system. Make sure that the details, such as the number of systems, companies, document types, frequency of receiving files, historical load, and opening balances, are taken into consideration.
  6. Identify the fields that are not captured by the interface and need extension in the SLT Replication Server interface.
  7. In SAP S/4HANA, for each company code, make sure that the necessary configurations are complete. Some of the configurations are as follows:
    • Accounting Period is Open
    • Number Ranges for Different Document Types
    • Tax Configuration
    • Extended Withholding Tax
    • Account Definition
    • Reconciliation Accounts
    • Clearing Accounts

Prerequisite Tasks for Loading NetSuite Data into SLT Replication Server

Before loading the data from your NetSuite instance to SLT Replication Server, you must load the data into the tables that you created in the source Oracle database during the installation of Transaction. For information about the tables, see "Install Source System-Specific Objects for Transaction" in the SAP Central Finance Transaction Replication by insightsoftware Installation and Configuration Guide.

To load the data from your NetSuite instance into Oracle database

  1. Open SAP Data Services Designer as an administrator, and connect to your repository.
  2. Run the NetSuite_% jobs in the NetSuite_DataLoad project. An initial load of transaction data is performed.
  3. Run the cdc_NetSuite_% jobs. The incremental data load process is run.
  4. Run the NSCDC_Set_Status_On job. The changed data is captured for transaction data.
  5. Run the NetSuite_trx_CDC job. The incremental transaction data is captured.
  6. When required, run the NetSuiteCDC_Set_Status_Off job to stop the incremental data load process.

Prerequisites for Loading Dynamics 365 Finance and Operations Data into SLT Replication Server

You must perform the following tasks before initiating the data load from your Dynamics 365 Finance and Operations Cloud instance to SLT Replication Server:

Import the following custom entities into the Azure SQL database using the Export option in Data Management Framework by referring to the links provided:

Note: You must import the custom entities before deploying the templates using Management Console.

  • ISWAccountingDistribution
  • ISWCompanyInfo
  • ISWCustConfirmJour
  • ISWCustInvoiceJour
  • ISWCustInvoiceTrans
  • ISWCustLedgerAccounts
  • ISWCustPackingSlipJour
  • ISWCustPackingSlipTrans
  • ISWCustSettlement
  • ISWCustTable
  • ISWCustTrans
  • ISWDimensionAttribute
  • ISWDimensionAttributeValue
  • ISWDimensionAttributeValueCombination
  • ISWDimensionAttributeValueSet
  • ISWDimensionFinancialTag
  • ISWDimensionFocusBalance
  • ISWDirPartyPostalAddress
  • ISWDirPartyTable
  • ISWDirPersonName
  • ISWEcoResProduct
  • ISWEcoResProductTranslation
  • ISWFiscalCalendarPeriod
  • ISWFiscalCalendarYear
  • ISWGeneralJournalAccountEntry
  • ISWGeneralJournalEntry
  • ISWHierarchyLinkTable
  • ISWHierarchyTreeTable
  • ISWInventDim
  • ISWInventItemInventSetup
  • ISWInventPosting
  • ISWInventTable
  • ISWInventTableModule
  • ISWLedger
  • ISWLedgerEntry
  • ISWLedgerEntryJournal
  • ISWLedgerEntryJournalizing
  • ISWLedgerJournalTable
  • ISWLedgerJournalTrans
  • ISWLogisticsElectronicAddress
  • ISWMainAccount
  • ISWMainAccountCategory
  • ISWMarkupTrans
  • ISWProjTable
  • ISWPurchLine
  • ISWPurchParmLine
  • ISWPurchParmTable
  • ISWPurchTable
  • ISWSalesLine
  • ISWSalesParmLine
  • ISWSalesParmTable
  • ISWSalesTable
  • ISWSmmActivities
  • ISWSubledgerJournalAccountEntry
  • ISWSubledgerJournalAccountEntryDistribution
  • ISWSubledgerVoucherGeneralJournalEntry
  • ISWTaxTrans
  • ISWTaxTransGeneralJournalAccountEntry
  • ISWTransactionLog
  • ISWTransactionReversalTrans
  • ISWUnitOfMeasureConversion
  • ISWVendInvoiceJour
  • ISWVendInvoiceTrans
  • ISWVendLedgerAccounts
  • ISWVendPackingSlipJour
  • ISWVendPackingSlipTrans
  • ISWVendPurchOrderJour
  • ISWVendSettlement
  • ISWVendTable
  • ISWVendTrans

Refer to the Prerequisite Tasks for Dynamics 365 Finance and Operations as source system section in the SAP Central Finance Master Data Replication by insightsoftware Installation and Configuration Guide and perform the steps.

Reference Fields for Reconciliation Objects

The following reference fields are added to the reconciliation objects such as the tables, views, and procedures. These reference fields are in addition to the already existing reference fields in the reconciliation reports. You can use these fields for your custom columns that are not available in the standard report by default.

  • 25 Dimensions:GLOBAL_REFERENCE_1 to GLOBAL_REFERENCE_25
  • 5 Measures:GLOBAL_REFERENCE_26 to GLOBAL_REFERENCE_30

For Level 1 to Level 5 tables, views, balance reports, and reconciliation reports, the following additional columns are added for the source and target fields:

  • For source:
    • Dimensions:SOURCE_REFERENCE_1 to SOURCE_REFERENCE_25
    • Measures:SOURCE_REFERENCE_26 to SOURCE_REFERENCE_30
  • For target:
    • Dimensions:TARGET_REFERENCE_1 to TARGET_REFERENCE_25
    • Measures:TARGET_REFERENCE_26 to TARGET_REFERENCE_30
  • For the exception views, only for source: 
    • Dimensions:GLOBAL_REFERENCE_1 to GLOBAL_REFERENCE_25
    • Measures:GLOBAL_REFERENCE_26 to GLOBAL_REFERENCE_30

To use the custom columns for the reconciliation related flows, you must modify the data flows having additional columns and map them to the dimensions and measures reference fields.

The following example explains how to use the custom columns.

To use the custom columns with the reference fields

  1. The following screenshot depicts the MAGNITUDE_STG_ONE data flow. The data flow is from the UFF_STG_TEMP table to the MAGNITUDE_STG_ONE table.
  2. Add TAX_RATE and TRANSLATION_DATE as the custom columns to the MAGNITUDE_STG_ONE table.
  3. The following view is in use for reconciliation support. Map the custom columns by loading the source TRANSLATION_DATE data in to the GLOBAL_REFERENCE_1 field and the source TAX_RATE data in to the GLOBAL_REFERENCE_26 field.

For updating column names for the reference fields in SAP Fiori, refer to Comparison Reports for Third-Party Systems Guide.

Load Data into SLT Replication Server

To load the data into SLT Replication Server, you will first load and map the master data using SAP MDG. Then, you will perform the historical transaction data load followed by the incremental data load.

To load the data into SLT

  1. Copy the MSC_Lookup_Data.xlsx file that you created during the configuration of SAP S/4HANA at the following location to your computer: \\SourceConnect\MasterMapping

    Note: For information about the configuration, see Configuring SAP S/4HANA for Data Load.

  2. In SAP Data Services Designer, connect to your repository, open the Master_Lookup_Load job pertaining to your source system, and perform the following:
    • In the MSC_MDFILE_CHK script file of the job, in the wait_for_file function, update the location of the MSC_Lookup_Data.xlsx file as \\SourceConnect\MasterMapping.
    • For MSC_MDFILE_EX, update the root directory as \\SourceConnect\Log&Reprocess\Log in which the log files must be generated.
  3. Update the values of the applicable global variables for the jobs pertaining to your source system before running the jobs. For information about the list of applicable global variables, see Global Variables in Appendix B.
  4. Run the Master_Lookup_Load job. The cache file for the MSC_MASTER_LOOKUP table is created.
  5. To load the customer, vendor, and general ledger account balances data into SLT Replication Server, run the <SourceSystem>_%_Balances or SC_Balances job.
  6. To load the historical data into SLT Replication Server:
    • Run the <SourceSystem>_Journals_OT or <SourceSystem>_Journals or SC_Journals job as applicable.

      • Important: For JD Edwards EnterpriseOne, before you run the job, run the R7404003 and R7403B001 reports in your source system in detail mode so that the F74412 worktable is populated with the customer and vendor balances data.

      Note: For information about running the job in parallel for multiple source systems, see Parallel Loading, Reprocessing of Flat File Data.

  7. For the incremental data load:
    1. Run the <SourceSystem>_CDC_Set_Status_On or SC_CDC_Set_Status_On job as applicable. The incremental data load is enabled for the <SourceSystem>_Journals_CDC and SC_Journals job respectively.
    2. Run the <SourceSystem>_Journals_CDC or SC_Journals job as applicable. The incremental data load is performed for journals that are posted in the source system.
    3. When required, run the <SourceSystem>_CDC_Set_Status_Off or SC_CDC_Set_Status_Off job as applicable. The incremental data load is stopped for the <SourceSystem>_Journals_CDC and SC_Journals job respectively.
  8. Verify the error log files that are generated at \\SourceConnect\Log&Reprocess\Log during the data load, and then, run the <SourceSystem>_Reprocess_Journals or SC_Reprocess job as applicable to reload the failed transactions. For more information, see Error Handling .

Load Reconciliation Data into Staging Database

After loading the transaction data into SLT Replication Server, you can load the Reconciliation data pertaining to your transactions into the staging database for Reconciliation reporting.

To load the Reconciliation data into the staging database

  1. Open SAP Data Services Designer as an administrator and connect to your repository.
  2. Update the values of the applicable global variables for the Reconciliation jobs pertaining to your source system. For the list of applicable global variables, see Global Variables in Appendix B.
  3. Run the following jobs in the <SourceSystem>_RECON_L1_L5 project:

    Note: If the staging database is SAP HANA, the <SourceSystem>_S4_FIN_Dataload and <SourceSystem>_S4_FIN_Dataload_CDC jobs are not applicable.

    • One-time data load
      • <SourceSystem>_S4_FIN_Dataload: The historical journal information based on the specified start date is loaded from SAP S/4HANA into the S4_FIN_TRX_STG table in the staging schema.
      • <SourceSystem>_Source_Doc_Info: The historical journal information is loaded from the source table in the source schema into the SOURCE_DOC_INFO table in the staging schema.
      • <SourceSystem>_Period_Balances_OT: The account balances data is loaded from the balances table in your source system into the <SourceSystem>_SOURCE_BALANCES tables in the staging schema.
    • <SourceSystem>_Account_Recon job
      • Run the job. The account balances data is loaded from the source system and SAP S/4HANA into the LEVEL5_RECON_GLBAL_STG table in the staging schema.
    • Incremental data load
      • <SourceSystem>_S4_FIN_Dataload_CDC: The journal information is incrementally loaded from SAP S/4HANA into the S4_FIN_TRX_STG table in the staging schema.
      • <SourceSystem>_SRCDOC_Set_Status_On: The incremental data load is enabled for the <SourceSystem>_Source_Doc_Info_CDC job.
      • <SourceSystem>_Source_Doc_Info_CDC: The journal information is incrementally loaded from the source table in the source schema to the SOURCE_DOC_INFO table in the staging schema.
      • <SourceSystem>CDC_PERIOD_Bal_Set_Status_On: The incremental data load is enabled for the <SourceSystem>_Period_Balances_CDC job.
      • <SourceSystem>_Period_Balances_CDC: The account balances data is incrementally loaded from the balances table in your source system into the <SourceSystem>_SOURCE_BALANCES tables in the staging schema.
      • <SourceSystem>_SRCDOC_Set_Status_Off: If required, run this job to stop the incremental data load for the <SourceSystem>_Source_Doc_Info_CDC job.
      • <SourceSystem>CDC_PERIOD_Bal_Set_Status_Off: If required, run this job to stop the incremental data load for the <SourceSystem>_Period_Balances_CDC job.
  4. Run the following jobs in the RECON_STG project:

    Note: If the staging database is SAP HANA , the SLT_FIN_STG and SLT_FIN_STG_CDC jobs are not applicable.

    • One-time data load 
      • SLT_FIN_STG: The historical data is loaded from the SLT tables into the SLT_DATA table in the staging schema.

        • Important: Before running the Recon_OT job, make sure that the MSC_CONFIG table contains the logical system and source system data in the following pattern:
        If the source is Oracle E-Business Suite and logical system is EBSPRD, enter the following:
        CONFIG_NAME: LOGSYSNAME
        CONFIG_VALUE: EBSPRD
        SOURCE_SYSTEM: EBS
        The value of CONFIG_NAME remains constant.

      • Recon_OT: The historical data is loaded into the LEVEL<1|2|3|4|5>_RECON%_STG tables in the staging schema.
    • Incremental data load
      • SLT_FIN_STG_CDC: The data is incrementally loaded from the SLT tables into the SLT_DATA table in the staging schema.
      • RECONCDC_Set_Status_On: The incremental data load is enabled for the RECON_CDC job.
      • RECON_CDC: The data is incrementally loaded into the LEVEL<1|2|3|4|5>_RECON%_STG tables in the staging schema.
      • RECONCDC_Set_Status_Off: If required, run this job to stop the incremental data load for the RECON_CDC job.
    • RECON_DEL_HIST
      • Run the job. The historical data is deleted in the LEVEL<1|2|3|4|5>_RECON%_STG tables based on the values provided for the Company, Fiscal Year, and/or Fiscal Period parameters during the job run.
    • For Oracle E-Business Suite, Oracle PeopleSoft, JD Edwards EnterpriseOne, MS Dynamics 365 Business Central (on-premises and cloud), MS Dynamics 365 Finance & Operations (cloud), MS Dynamics NAV, MS Dynamics AX, Infor LN (on-premises), and Flat file/Universal Connector:
      • RECON_L1_L3: Loads L1-L3 reconciliation reports simultaneously, with the flexibility to refresh specific levels of reports as needed.
      • RECON_L4_L5: Loads L4 and L5 reconciliation reports simultaneously, with the flexibility to refresh specific levels of reports as needed.

After you initiate the job runs, depending on the delay specified in the G_SLEEP variable of the RECON_CDC job and the run duration of the jobs, the Reconciliation information will be available in the Reconciliation reports.

Additionally, for information on the documents that are not posted into SAP S/4HANA due to data load-related issues in the staging database, you can see the Exception report. For information on this report and other Reconciliation reports, see Comparison Reports for Third-Party Systems Guide.

Load Transaction Data into SAP S/4HANA for Drilldown Reporting

You can load the detailed information about transactions from your source system into SAP S/4HANA for Drilldown reporting.

To load the transaction data for Drilldown reporting

  1. Open SAP Data Services Designer as an administrator, and connect to your repository.
  2. Run the following jobs depending on your data load requirements:
    • One-time data load
      • Run the <SourceSystem>_OT% jobs pertaining to your source system. The historical data is loaded for the Drilldown user that you created during the setup of Drilldown.
    • Incremental data load
      • Run the <SourceSystem>_CDC_Set_Status_On job by specifying the <SourceSystem>_CDC% job name in the $G_SUBSCRIPTION global variable. The incremental data load is enabled for the specified job.
      • Run the <SourceSystem>_CDC% job. The incremental data load is performed for journals that are posted in the source system.
      • When required, run the <SourceSystem>_CDC_Set_Status_Off job by specifying the <SourceSystem>_CDC% job name in the $G_SUBSCRIPTION global variable. The incremental data load process is stopped for the specified job.

Load Data Using SyncBack

After importing the jobs and updating the data store settings in the SAP Data Services repository, SyncBack enables you to load the payments and receipts data from SAP S/4HANA into the following source systems:

  • Oracle E-Business Suite
  • JD Edwards EnterpriseOne
  • Microsoft Dynamics AX

  • Microsoft Dynamics 365 NAV

  • Microsoft Dynamics 365 Business Central (On-Premises)

  • QAD
  • 1C:Enterprise
  • Flat file

To load the data from SAP S/4HANA into the source system

  1. Open SAP Data Services Designer as an administrator, and connect to your repository.
  2. For all staging databases except SAP HANA, perform the following data loads depending on your requirements:
    • One-time data load
      1. Run the S4_<SourceSystem>_AP_Payment_Trx job. The historical payments data in SAP S/4HANA is loaded into the AP_FIN_PYMT_TRX table in the staging database.
      2. Run the S4_<SourceSystem>_AR_Payment_Trx job. The historical receipts data in SAP S/4HANA is loaded into the AR_FIN_PYMT_TRX table in the staging database.
    • Incremental data load
      1. Run the S4_<SourceSystem>STG_Set_Status_On job by specifying the
        S4_<SourceSystem>_AP|AR_Payment_Trx_CDC job name in the $G_SUBSCRIPTION global variable. The incremental data load is enabled for the specified job.
      2. Run the S4_<SourceSystem>_AP_Payment_Trx_CDC job. The payments data in SAP S/4HANA is incrementally loaded into the AP_FIN_PYMT_TRX table in the staging database.
      3. Run the S4_<SourceSystem>_AR_Payment_Trx_CDC job. The receipts data in SAP S/4HANA is incrementally loaded into the AR_FIN_PYMT_TRX table in the staging database.
      4. When required, run the S4_<SourceSystem>STG_Set_Status_Off job by specifying the S4_<SourceSystem>_AP|AR_Payment_Trx_CDC job name in the
        $G_SUBSCRIPTION global variable. The incremental data load process is stopped for the specified job.
  3. For the staging database as SAP S/4HANA:
    • Run the S4_<SourceSystem>_AP_Payment_Trx_HANA and S4_<SourceSystem>_AP_Payment_Trx_HANA jobs for one-time data load.
    • Run the S4_<SourceSystem>_AR_Payment_Trx_CDC_HANA and S4_<SourceSystem>_AR_Payment_Trx_CDC_HANA jobs for the incremental data load.
  4. In the $SAP_COMPANY global variable of the S4%OT and S4%CDC jobs, specify the company code through which the payment or receipt details of the company must be extracted.
  5. In the $LOGICAL_SYSTEM global variable, specify the logical system value such as UFFPRD or EBSPRD.
  6. Then, run the following jobs depending on your data load requirements:
    • One-time data load
      1. Run the S4_<SourceSystem>_AP_Payment_OT job. The historical payments data in SAP S/4HANA is loaded into the vendor payment-specific table in the source database or a file.
      2. For both one-time and incremental data loads, the historical payments data in the S4_AP_Payment_Info view is loaded into the vendor payment-specific table in the source database or a file.
      1. Run the S4_<SourceSystem>_AR_Payment_OT job. The historical payments data in SAP S/4HANA is loaded into the customer receipt-specific table in the source database or a file. For both one-time and incremental data loads, the historical payments data in the S4_AR_Receipt_Info view is loaded into the customer receipt-specific table in the source database or a file.
    • Incremental data load
      1. Run the S4_<SourceSystem>CDC_Set_Status_On job by specifying the
        S4_<SourceSystem>_AP|AR_Payment_CDC job name in the $G_SUBSCRIPTION global variable. The incremental data load is enabled for the specified job.
      2. Run the S4_<SourceSystem>_AP_Payment_CDC job. The payments data in SAP S/4HANA is incrementally loaded into the vendor payment-specific table in the source database or a file.
      3. Run the S4_<SourceSystem>_AR_Payment_CDC job. The receipts data in SAP S/4HANA is incrementally loaded into the customer receipt-specific table in the source database or a file.
      4. When required, run the S4_<SourceSystem>CDC_Set_Status_Off job by specifying the S4_<SourceSystem>_AP|AR_Payment_CDC job name in the $G_SUBSCRIPTION global variable. The incremental data load process is stopped for the specified job.
  7. (For Microsoft Dynamics 365 NAV and Business Central On-Premises) Run the following jobs in the SC_Syncback project:
    1. S4_AP_PAYMENT_TRX: The historical payments data in the staging database is loaded into the vendor payment-specific table in the source database or a file.
    2. S4_AR_PAYMENT_TRX: The historical payments data in the staging database is loaded into the customer receipt-specific table in the source database or a file.
    3. S4_SET_STATUS_ON: The incremental data load is enabled for the AP/AR job.
    4. S4_SET_STATUS_OFF: The incremental data load process is stopped for the AP/AR job.
  8. Schedule and run the SyncBack APIs that were registered during the installation of SyncBack. The payments are created and the corresponding invoices are cleared in the source system.

MSC_LOOKUP_DATA Entries for Additional Payment Scenarios

For Oracle E-Business Suite, make sure that the MSC_LOOKUP_DATA Excel workbook is updated with the accurate source-to-target mappings by referring to the following screenshots.

Note: These mappings are required for the data to flow back from SAP S/4HANA to the source system during SyncBack.

The following mappings for Transaction Replication contain the Source_Value1 field which indicates the payment type in the source system and the Target_Value1 field which indicates the payment type in SAP S/4HANA .

The following mappings for Syncback contain the Source_Value1 field which indicates the payment type in SAP S/4HANA and the Target_Value1 field which indicates the payment type in the source system.

Error Handling

After you run the jobs for loading the transaction data, Transaction validates and logs errors for documents that cannot be loaded into SLT Replication Server. The objective of error handling and reprocessing is to reprocess the documents that failed during the transaction replication process. The document can fail in the following scenarios:

  • Due to non-extraction
  • Due to validation failure
  • Due to pre-AIF validation failure
  • Due to failure in AIF

Reprocessing Failed Documents

This section contains information about reprocessing the failed documents that are identified through the error handling process. The data reprocessing framework includes the following two jobs:

  • Reprocess
  • Clean up

The data reprocessing framework is applicable for the following source systems:

  • Oracle E-Business Suite
  • JD Edwards EnterpriseOne
  • Oracle PeopleSoft
  • UC/Flat file
  • Microsoft Dynamics AX
  • Microsoft Dynamics NAV
  • Microsoft Dynamics GP
  • Microsoft Dynamics 365 Finance and Operations
  • Microsoft Dynamics 365 Business Central (on-premises and cloud)
  • Infor LN (on-premises)

Prerequisites

You must have access to the following job and .csv file for reprocessing the failed documents:

  • <SourceSystem>_Reprocess_Journals job

      The selection parameters for the reprocessing job are as follows:

    • $G_PROCESS_MISSING_DOCS: Enter Y to reprocess the documents that were missed during extraction. It is supposed that the missing document details are entered in the .csv file located in the shared location.
    • $G_PROCESS_VALIDATION_ERRORS: Enter Y to reprocess all the UNIQUE_ID fields that failed during insightsoftware validation process and now exist in the <SOURCE SYSTEM>_STG_ERROR_LOG table.
    • $G_PROCESS_AIF_UIDs: Enter Y to process all the Unique_ID fields placed in the .csv file. It is supposed that all the documents having failed Unique_IDs in the AIF are placed in the .csv file.
    • $GARCH_FLAG: Enter Y to archive the data of Magnitude_STG_One and FIN% tables before deletion.
    • $G_USE_AUTO_INPUT: Enter Y to automatically reprocess all the documents in the stage error logs and AIF tables (whichever is applicable).
    • $G_USE_EXCEL_INPUT: Enter Y to select the documents to reprocess using Excel input parameters.

      • Note: For the Process_Missing_Docs and Process_AIF_UIDs options to function accurately, you must enter the JE_HEADER_ID field for the applicable tables. For JD Edwards EnterpriseOne, enter the DOC_BATCH and DOC_NUMBER fields.

  • SC_Reprocess job for Oracle E-Business Suite, JD Edwards EnterpriseOne, Oracle PeopleSoft, Microsoft Dynamics GP, Microsoft 365 Dynamics Finance & Operations, Microsoft Dynamics 365 Business Central (on-premises and cloud), Infor LN (on-premises), and Flat file. Refer to the global variables in Appendix B of the SAP Central Finance Transaction Replication by insightsoftware Administrator Guide for related information.
  • <SourceSystem>_REPROCESS_FILE.csv file
  • <SourceSystem>_JOURNALS_OT job
  • SC_Journals job for Oracle E-Business Suite, JD Edwards EnterpriseOne, and Flat file.

To reprocess documents

  1. Make sure that the failed documents are corrected and ready for reprocessing.
  2. Select the <SOURCE SYSTEM>_Reprocess_Journals or SC_Reprocess job in <SourceSystem> Journals and go to the Execution Properties tab.
  3. Toggle the required global variable to ‘Y’ considering the scenario in which the documents have failed
    • Stage and FIN validation failure, Custom Validation failure - $G_Process_Validation_Errors (‘Y’/’N’) – ‘Y’
    • AIF failure at CFIN - $G_Process_AIF_UIDs (‘Y’/’N’) – ‘Y’
    • Documents missing during extraction from Source - $G_Process_Missing_Docs (‘Y’/’N’) – ‘Y’

    The documents are picked for reprocessing in one of the following ways:

    • Automated document capture: Setting $G_USE_AUTO_INPUT as Y will automatically reprocess all the documents in the stage error logs and AIF tables (whichever is applicable).
    • Manual document capture: Setting $G_USE_EXCEL_INPUT as Y will allow you to choose the documents to reprocess using Excel input parameters. You must update and save the Excel template in the designated path so that the <SourceSystem>_Reprocess_Journals job can utilize it.
  4. Run the <SourceSystem>_Reprocess_Journals or SC_Reprocess job after selecting the appropriate toggles.

As a result:

  • All the documents picked up by the <SourceSystem>_Reprocess_Journals job will be cleaned up automatically, ensuring that there is no occurrence of unique constraint error.
  • All the cleaned up documents will be archived for reference.
  • No manual clean up is required at the Stage, FIN, or SLT levels.
  • Selected records will be moved to Records to reprocess table post which, the documents get reprocessed.

Cleaning up Documents

In transaction replication, the cleanup job is essential for maintaining the efficiency and performance of the replication process. You can choose between a manual or automated cleanup process.

Automated cleanup process:

  1. Enter the global variables for the SCT_CLEANUP job. For details on the global variables, refer to Appendix B.

    Important: Only records that meet all input parameter criteria will be selected for deletion during the automated cleanup.

    Note: You can enter multiple values for Company Code and Posting Period at the global variable level, separated by commas. The maximum number of values is set to 30, which can be adjusted if needed.

  2. Trigger the SCT_CLEANUP job.

Manual cleanup process:

  1. Download the FF_SC_CLEANUP.csv file located at SCT/Common_Mappings/SCT_CLEANUP.
  2. Fill in the columns with the required information: SOURCE_SYSTEM, BATCH, COMPANY_CODE, FISCAL_YEAR_DOC_TYPE, FROM_POSTED_DATE, TO_POSTED _DATE, and RUN_ID.
  3. Upload the CSV file containing the documents to be deleted.
  4. Trigger the SCT_CLEANUP job.

Upon completion, the data in SCT_STG_ARCHIVE is automatically backed up. You can access this backup using the generated .csv file located at the path specified in the ARCHIVE_FILEPATH variable.

Implementing Custom Validations

You can add custom validations to FIN% tables for early error detection in AIF, facilitating timely reprocessing. The custom validations are applicable for the following source systems:

  • Oracle E-Business Suite
  • JD Edwards EnterpriseOne
  • Oracle PeopleSoft
  • UC/Flat file
  • Microsoft Dynamics AX
  • Microsoft Dynamics NAV
  • Microsoft Dynamics GP
  • Dynamics 365 Business Central (on-premises and cloud)
  • Dynamics 365 Finance and Operations
  • Infor LN (on-premises)

To reprocess failed documents

  1. Based on the category of failure, the system reprocesses the documents based on either the document or the unique identifier for the identified category of failure.

    • For the documents that were created in the source system but missed the extraction, the system:

      • populates the reprocessing file, <SOURCE SYSTEM>_REPROCESS_FILE.csv located at \\SourceConnect\Log&Reprocess\Reprocess.
      • uses the Level 1 Reconciliation Report and searches for the documents that were missed during extraction
      • populates the <SOURCE SYSTEM>_Reprocess_file.csv spreadsheet placed in the shared folder with the details as shown in the following screenshot:
    • For the documents that failed in the stage validations, you must resolve the issues that caused the validation failure and then select this option. For this process, you do not have to populate the .csv file because the system fetches the information from the staging table.
    • For the documents that failed at the pre-AIF validation stage, you must perform the following steps:
      1. Download the Custom AIF-FIN simulator.xls file located at <SetupFolder>\<SourceSystem.
      2. The Excel file consists of the following three sheets:
        • Instructions
        • Editable_Default_validations
        • Custom Validations
      3. A set of common AIF validations are placed in the Editable_Default_Validations sheet. As a part of the current release:
        • Fields in yellow are editable.
        • ‘Check’ and ‘condition’ columns are non-editable.
      4. Enter the required fields in the Editable_Default_Validations sheet. For example, if a document with BUSINESS_ACTIVITY has field value other than ‘RTGE’, ‘TRFG’, it is assumed to fail in SAP S/4HANA.
      5. Enter the appropriate FIN table, Column, Field Values, and select Enabled as Status.
      6. Go to Custom Validations sheet and edit the yellow customisable fields by entering the customized values.
      7. After entering all the validations, place the updated file in the designated path (to be updated).
      8. Before running the <SOURCE SYSTEM>_JOURNALS_OT or SC_Journals job, select ‘Yes’ in the custom validation section of the run parameters. The documents failing at the pre-AIF validation will be recorded in the error logs, which can be reprocessed using the reprocess or one-time job post error handling.
    • For the documents that failed during the AIF process, the system reprocesses the data in the source system, and:
      • populates the reprocessing file, <SOURCE SYSTEM>_REPROCESS_FILE.csv located at \\SourceConnect\Log&Reprocess\Reprocess.
      • uses the Level 1 Reconciliation Report and searches for the documents that were missed during extraction
      • populates the <SOURCE SYSTEM>_Reprocess_file.csv spreadsheet placed in the shared folder with the details as shown in the following screenshot:
  2. As reprocessing is performed from the source system, the system performs a data cleanup and deletes the data from the following tables :
    • MAGNITUDE_STG_ONE Table
    • FIN% tables
    • SLT tables

    The data cleanup is done based on the data in the RECORDS_TO_ REPROCESS_STG table data and loads the data into this table. The data is deleted from SLT tables followed by MAGNITUDE_STG_ONE and FIN% tables. The deletion of MAGNITUDE_STG_ONE and FIN% table data also includes archiving the data into archive tables.

    Note: Archiving the data is an optional step and it depends on the user selection while running the reprocessing job.

  3. The system loads the data by running the POP_REPROCESS_DATA procedure to fetch the data from the staging table and load it into the source table of the corresponding module. The Records_reprocess_src table contains data related to all the reprocessing documents.

To implement custom validations for JD Edwards EnterpriseOne and Flat file sources

  1. Download the Custom AIF-FIN simulator.xls file located at <SetupFolder>\<SourceSystem.
  2. The Excel file consists of the following three sheets:
    • Instructions
    • Editable_Default_validations
    • Custom Validations
  3. A set of common AIF validations are placed in the Editable_Default_Validations sheet. As a part of the current release:
    • Fields in yellow are editable.
    • ‘Check’ and ‘condition’ columns are non-editable.
  4. Enter the required fields in the Editable_Default_Validations sheet. For example, if a document with BUSINESS_ACTIVITY has field value other than ‘RTGE’, ‘TRFG’, it is assumed to fail in SAP S/4HANA.
  5. Enter the appropriate FIN table, Column, Field Values, and select Enabled as Status.
  6. Go to Custom Validations sheet and edit the yellow customisable fields by entering the customized values.
  7. After entering all the validations, place the updated file in the designated path (to be updated).
  8. Before running the <SourceSystem>_JOURNALS_OT or SC_Journals job, select ‘Yes’ in the custom validation section of the run parameters. The documents failing at the pre-AIF validation will be recorded in the error logs, which can be reprocessed using the reprocess or one-time job post error handling.

To reprocess failed documents at regular intervals

The failed documents identified through the error handling process are reprocessed by running the EBS_Reprocess_Journals job. To reprocess the documents at regular intervals, perform the following steps:

  1. Open SAP Data Services Designer as an administrator and select the relevant batch.
  2. Select the Batch Job Configuration tab and locate the EBS_Reprocess_Journals job.
  3. As a prerequisite before scheduling the reprocessing job, ensure that there is no existing one-time job currently being run.
  4. Select Add to add a new schedule for the EBS_Reprocess_Journals job as shown in the following screenshot:
  5. Enter an appropriate name and provide other necessary details for the required schedule as shown in the following screenshot:
  6. Configure the global variables and select Apply to save the scheduler details. The schedule is created, however, it is not activated.
  7. Go to Batch > Batch Job Schedules and check that the newly added schedule is in an inactive state.
  8. Select the schedule and select Activate. The schedule is activated.
  9. Check the admin console to monitor the schedule.
  10. The job is triggered at the scheduled time and run as per the set configurations.

    The documents are reprocessed at regular intervals.

To run the AIF error validation in SAP Data Services

  1. Set the value of the global variable $G_VALIDATE_AIF as 'Y' if you want to run the AIF validation. By default, the validation is run for the module selected to process one-time job as the global variable is reused here.

  2. The object key and header ID are taken from the staging tables based on the Custom FIN-AIF Validator.xlsx located at \TransactionReplication.zip\SCT\<SourceSystem>\Oracle_Source for Oracle E-Business Suite and JD Edwards EnterpriseOne. For the Flat file source, the location is \TransactionReplication.zip\SCT\UFF. There are pre -defined validations set on columns which can either be enabled or disabled by the user.

  3. A procedure named AIF_VALIDATION_PROC is used to validate the data on the stage tables.

  4. The failed records are logged in to <SourceSystem>_STG_ERROR_LOG.txt.

  5. The SAP Data Services job automatically generates the report with the record details and error reason.

To download error report at the staging level

The SAP Central Finance Master Data Replication by insightsoftware for SAP S/4HANA has inbuilt capabilities to generate an error log report for the records that failed at the stage level.

For example, when you run a one-time job, if there are any failures during validation process, they are captured in a table, and then downloaded as a report in the directory.

The flow which generates the error report:

The following code generates the name of the error report file.

The location of the file is maintained at \SourceConnect\Log&Reprocess\Log.

As the job completes running, the log is generated at the designated location along with the timestamp.

Following is a sample report with details.

The .txt report contains all the major fields required to uniquely identify a record and along with it comes the description due to which the record failed.

Import the .txt error log file from the \SourceConnect\Log&Reprocess\Log folder for analysis.

Was this article helpful?

We're sorry to hear that.