SAP S/4HANA, SAP ERP

AI-based Invoice Data Capture Integration with S/4HANA

Introduction

I have discovered recently Rossum.ai, a Data Capture tool based on artificial intelligence. This is not OCR, but an AI engine which detects data fields and annotate scanned documents.

I really like the freemium approach, where you can test and play with in order to evaluate. This is quite uncommon, as Invoice Capture products in SAP landscape are usually untouchable before you actually start commercial discussions with sales representatives.

Also Read: SAP S/4HANA Sourcing and Procurement Certification Preparation Guide

Their API is simple, yet efficient and well-documented, and you can start playing with SAP integration in no time.

We will implement a simple demo use case as follows:

  • A purchase order is created in S/4HANA (simple as 2 item lines, no need for goods receipt before invoice)
  • A paper supplier invoice is received for this PO (simple invoice with no matching difference)
  • Invoice is scanned into an image document
  • Image is uploaded in Rossum cockpit
  • Automatic annotations occurs
  • Annotations are manually checked and validated
  • Invoice data is sent automatically to S/4HANA and an Incoming Invoice document is posted

The final process will look like:

In order to accomplish it, we will have to go through these implementation steps:

  1. Have all our accounts and settings ready in both Rossum and S/4 (of course)
  2. Analyze and understand API message payloads with a REST client (postman)
  3. Extend Rossum schema to add a custom field holding SAP supplier code
  4. Implement extension hook “validate”: simple supplier matching to retrieve SAP supplier code from name
  5. Implement extension hook “save”: transfer invoice to S/4HANA

Setup

Rossum.ai

Our account is already usable out of the box after registration: we can upload invoices and check automatic annotations. No overwhelming setup or training is required, which is really huge time saving.

In order to use API and have access to extended option, we will install command line tool elisctl.

SAP S/4HANA

We need to make sure OData services for Partners et Supplier Invoices are activated in /IWFND/MAINT_SERVICES: API_BUSINESS_PARTNER and API_SUPPLIERINVOICE_PROCESS_SRV. I am not going to provide much more details, I assume we know how to use them.

API: understanding annotation structure

Rossum API is simple to use. Every request must be authorized by a token, which is provided as we login:

  • Login and retrieve authentication token

Field value key in response payload is our authorization token, and must be provided in our request headers.

  • Retrieve queues: in our case only one is available
  • Retrieve annotations from queue

Annotation structure is exactly as on validation screen:

Extend schema with custom field “SAP supplier”

Schema can be extended with API calls or with elisctl. I have no preference so far and we will use command lines: at least it will (very slightly) simplify upload/download.

First, logon to our account using elisctl:

Welcome to the elisctl interactive mode. Start with `help` and `configure`.
elis> configure
API URL [https://api.elis.rossum.ai]: 
Username: xxxx@yyy
Password: ******

Detected fields and annotations are declared in a schema. We need to extend our current schema so to make available one extra field in the annotation. We start by downloading it.

elis> queue list
   id  name                    workspace  inbox                           schema    users  connector                                      webhooks
-----  --------------------  -----------  ----------------------------  --------  -------  ---------------------------------------------  ----------
28056  Received EU invoices        25714  xxxxxxxxxxxxx@elis.rossum.ai    240106    35005  https://api.elis.rossum.ai/v1/connectors/2998

elis> schema get 240106 -O testschema.json

Then change it by adding a new datapoint.

Once done, We upload it back.

elis> schema update 240106 testschema.json

Our new field is now available and visible in the settings

And voila. It is directly available as an annotation field, and of course we can add as many custom fields as required: SAP material code, SAP company code, and much more. We will keep our demo simple though and have only one custom field.

The new added field is just a placeholder for now: Rossum is not going to magically determine my backend supplier code whereas its number does not appear on invoice. We could as well let it entirely manual, expecting end-user to fill it during validation step, but this would make no sense. This is where Rossum extension comes into play.

In the next step, We will declare a custom hook which will be triggered during validation: the so-called validate extension. Our extension will then call OData service API_BUSINESS_PARTNER and match a supplier code with the supplier name as recognized by Rossum. Actually we am going to reuse the partner search function we have created for our chatbot.

Implement custom hook validate

Declare a custom plugin: we use endpoint connector to post a new extension, which is no more than a name and an url. Pay attention to authorization token: it is our responsibility to authorize only request providing this token (up to us to decide on the token value/validation in our code). Though not recommended, we could as well fill any dummy value here and ignore authorization token in our custom code.

Now our extension is declared, we need to implement endpoint function /validate.

As explained in my earlier post, we will simply implement our hooks using Firebird Cloud Functions. Hassle-free service. We could do the same with AWS lambda.

The incoming request will post a payload message like this:

{
  "meta": {
    "document_url": "https://api.elis.rossum.ai/v1/documents/6780",
    "arrived_at": "2019-01-30T07:55:13.208304Z",
    ... ...
    "rir_poll_id": "54f6b9ecfa751789f71ddf12"
  },
  "content": [
        {
            "id": 133915359,
            "url": "https://api.elis.rossum.ai/v1/annotations/1166616/content/133915359",
            "children": [...],
            "category": "section",
            "schema_id": "invoice_info_section"
        },
        {
            "id": 133915368,
            "url": "https://api.elis.rossum.ai/v1/annotations/1166616/content/133915368",
            "children": [...],
            "category": "section",
            "schema_id": "payment_info_section"
        },
        {
            "id": 133915378,
            "url": "https://api.elis.rossum.ai/v1/annotations/1166616/content/133915378",
            "children": [...],
            "category": "section",
            "schema_id": "amounts_section"
        },
        {
            "id": 133915386,
            "url": "https://api.elis.rossum.ai/v1/annotations/1166616/content/133915386",
            "children": [
                {
                    "id": 137835747,
                    "url": "https://api.elis.rossum.ai/v1/annotations/1201234/content/137835747",
                    "content": {
                        "value": "MyDearSupplier",
                        "page": 1,
                        "position": [
                            889.0,
                            168.0,
                            1067.0,
                            210.0
                        ],
                        "rir_text": "MyDearSupplier",
                        "rir_position": [
                            889.0,
                            168.0,
                            1067.0,
                            210.0
                        ],
                        "rir_confidence": 0.6798204155690549,
                        "connector_position": null,
                        "connector_text": null
                    },
                    "category": "datapoint",
                    "schema_id": "sender_name",
                    "validation_sources": [],
                    "time_spent": 0.0
                },
                ...,
                {
                    "id": 137835750,
                    "url": "https://api.elis.rossum.ai/v1/annotations/1201234/content/137835750",
                    "content": {
                        "value": "",
                        "page": null,
                        "position": null,
                        "rir_text": "",
                        "rir_position": null,
                        "rir_confidence": null,
                        "connector_position": null,
                        "connector_text": null
                    },
                    "category": "datapoint",
                    "schema_id": "sap_supplier",
                    "validation_sources": [],
                    "time_spent": 0.0
                },
                ...
            ],
            "category": "section",
            "schema_id": "vendor_section"
        },
        {
            "id": 133915396,
            "url": "https://api.elis.rossum.ai/v1/annotations/1166616/content/133915396",
            "children": [...],
            "category": "section",
            "schema_id": "other_section"
        },
        {
            "id": 133915398,
            "url": "https://api.elis.rossum.ai/v1/annotations/1166616/content/133915398",
            "children": [...],
            "category": "section",
            "schema_id": "line_items_section"
        }
    ]
}

As shown in this sample, we will find supplier name under payload.content[3].children[0] and SAP supplier code under payload.content[3].children[1]. Of course we are not going to fetch by index directly (too much bug prone), but in our simple custom code, we will loop over all datapoints in seek of supplier name by checking schema_id. Once we have it, we will call our backend OData service to match with SAP supplier number. Finally, we will update our custom datapoint SAP supplier code with the correct number by responding with message:

{
  "messages": [
    {
      "content": "SAP Supplier <name> is matched with SAP number <sap_code>",
      "id": "<datapoint_id (=137835750 in sample)>"
      "type": "info"
    }
  ],
  "updated_datapoints": [
    {
      "id": "<datapoint_id (=137835750 in sample)>",
      "value": "<sap_code>"
    }
  ]
}

Sample implementation of /validate:

exports.validate = functions.https.onRequest((request, response) => {
    console.log(request.body);

    //response.sendStatus(200);
    return parseRossumContent(request.body)
        .then((content:any) => {
            return response.json({
                messages: [
                    {
                        content: "OK",
                        type: "info",
                        id: `${content.id}`
                    }
                ],
                updated_datapoints: [
                    content
                ]
            });
        })
        .catch((error: any) => {
            // handle error
            console.log(error);
        })
});

where parseRossumContent function is:

function parseRossumContent(data:any):Promise<any> {

    // Vendor name
    let sender_name:string = ''

    // SAP supplier annotation id
    let sap_supplier_schema_id:string = ''

    // Loop over all content lines until vendor_section
    data.content.some((section:any) => {

        // Found, now check its children
        if(section.schema_id === 'vendor_section') {

            // Loop over vendor_section.children until sender_name and sap_supplier
            section.children.some((subsection:any) => {
                if(subsection.schema_id === 'sender_name') {
                    sender_name = subsection.value;
                }
                else if(subsection.schema_id === 'sap_supplier') {
                    sap_supplier_schema_id = subsection.id
                }
                return (sender_name !== '' && sap_supplier_schema_id != '')
            })
            return true
        }
        return false
    })

    // At this stage, both sender_name and sap_supplier_schema_id are known
    return searchSupplier(sender_name)
        .then((sap_code:any) => {
            return {
                id: sap_supplier_schema_id,
                value: sap_code
            }
        })
}

Implement custom hook save

As with the previous section, we implement endpoint /save. In this part, we will integrate Rossum data as supplier invoice in S/4HANA.

Incoming payload has the same structure as for /validate, and no response payload is expected in case of successful processing, just HTTP status code 204.

To create an invoice in S/4 is as easy as posting OData service API_SUPPLIERINVOICE_PROCESS_SRV with the following payload:

{
    "CompanyCode": "1710",
    "DocumentDate": "/Date(1574208000000)/",
    "PostingDate": "/Date(1574208000000)/",
    "SupplierInvoiceIDByInvcgParty": "Vendor invoice number",
    "InvoicingParty": "17300032",
    "DocumentCurrency": "USD",
    "InvoiceGrossAmount": "120000.00",
    "to_SuplrInvcItemPurOrdRef": [
        {
            "SupplierInvoiceItem": "1",
            "PurchaseOrder": "4500000004",
            "PurchaseOrderItem": "10",
            "Plant": "1710",
            "TaxCode": "I1",
            "DocumentCurrency": "USD",
            "SupplierInvoiceItemAmount": "120000.00",
            "PurchaseOrderQuantityUnit": "PC",
            "QuantityInPurchaseOrderUnit": "30",
            "PurchaseOrderPriceUnit": "PC",
            "QtyInPurchaseOrderPriceUnit": "30"
        }
    ]
}

Our /save function is not very complex: we start by parsing annotation in a kind of key-value structure, then we call API in 2 steps, first GET to retrieve CSRF token and cookie, then POST to actually create the supplier invoice.

exports.save = functions.https.onRequest((request, response) => {
    console.log(request.body);
    const parsedContent = parseRossumContentForSave(request.body)

    const url = "https://yourserver:443/sap/opu/odata/sap/API_SUPPLIERINVOICE_PROCESS_SRV/A_SupplierInvoice";

    // Axios request options
    let options = {
        headers: {
            "Authorization": "Basic xxx",
            "Content-Type": "application/json",
            "Accept": "application/json",
            "x-csrf-token": "fetch",
        },
        params: {
            'sap-client': '100',
            '$format': 'json',
            '$top': 0
        },
        xsrfHeaderName: 'x-csrf-token',
        withCredentials: true
    };

    // Store invoice date in javascript format
    const invoiceDate:Date = new Date(parsedContent.invoice_info_section.date_issue)

    // Declare items to be filled, total amount will be the sum of item amounts
    let items:any[] = []
    let total = 0
    let itemNum = 0

    // Prepare items
    parsedContent.items.forEach((item:any) => {
        itemNum++
        total += Number(item.item_amount_total.replace(/\s+/g, ''))
        items.push({
            SupplierInvoiceItem: itemNum.toString(),
            PurchaseOrder: parsedContent.invoice_info_section.order_id,
            PurchaseOrderItem: (itemNum*10) .toString(),
            Plant: "1710",
            TaxCode: "I1",
            DocumentCurrency: parsedContent.amounts_section.currency.toUpperCase(),
            SupplierInvoiceItemAmount: item.item_amount_total.replace(/\s+/g, ''),
            PurchaseOrderQuantityUnit: "PC",
            QuantityInPurchaseOrderUnit: item.item_quantity.replace(/\s+/g, ''),
            PurchaseOrderPriceUnit: "PC",
            QtyInPurchaseOrderPriceUnit: item.item_quantity.replace(/\s+/g, '')
        })
    })

    // First GET request is to get CSRF token
    axios.get(url, options)
        .then((res:any) => {
            console.log("Get OK")
            console.log(res)

            let invoice_json = {
                CompanyCode: "1710",
                DocumentDate: `/Date(${invoiceDate.valueOf()})/` ,
                PostingDate: `/Date(${invoiceDate.valueOf()})/`,
                SupplierInvoiceIDByInvcgParty: parsedContent.invoice_info_section.invoice_id,
                InvoicingParty: parsedContent.vendor_section.sap_supplier,
                DocumentCurrency: parsedContent.amounts_section.currency.toUpperCase(),
                InvoiceGrossAmount: total.toString(),
                
                to_SuplrInvcItemPurOrdRef: items
            }

            // CSRF token in header must be accompanied with Cookie. Not sure this is the best way to manage it ...
            let cookie = ""
            res.headers['set-cookie'].forEach((val:string) => {
                cookie += val.split(";")[0] + ";"
            })

            return axios.post(url,invoice_json, {
                headers: {
                    "Authorization": "Basic xxx",
                    "Content-Type": "application/json",
                    "Accept": "application/json",
                    "x-csrf-token": res.headers['x-csrf-token'],
                    'Cookie': cookie,
                },
                params: {
                    'sap-client': '100'
                },
                xsrfHeaderName: 'x-csrf-token',
                withCredentials: true
            })
        })
        .then(response.status(204))
        .catch((error:any) => {
            console.log(error.response)
        })
});

Leave a Reply

Your email address will not be published. Required fields are marked *