Creating ABAP Dataframes for SAP PAL

[thomas_mller13]

Machine learning algorithms often need large datasets for training. E.g. the MNIST dataset consists of 70.000 records. Each record represents a little greyscale image of size 28x28 pixels of a handwritten number and the respective number as integer value ,  i.e. 28x28 =  784 integer values in the range interval [0,255] and one integer value for the number value.

Assume you want to train a neural network in PAL using AMDP (ABAP managed database procedures) in order to predict the number. For this task you could e.g. use the PAL algorithm "Multilayer Perceptron", which is a neural network. In order to supply the input neurons with data a dataframe is needed, i.e. an ABAP-DDIC structure containing the following fields:

ID = Identifier of the record

Y = dependent variable = predicted (classified) value

X1 = Independent variable 1 = greyscale value of pixel 1

X2 = Independent variable 2 = greyscale value of pixel 2

...

X784 = Independent variable 784 = greyscale value of pixel 784.

This data structure and it's corresponding table will contain the input data for the neural network. Of course it is possible to create such a DDIC structure by hand. But creating a structure with 786 components costs a lot of time. Since most of the component names can be derived from other data the structure can be generated automatically. The following ABAP shows the code in order to create a DDIC structure and a DDIC table type for the MNIST example:   

*&---------------------------------------------------------------------*
*& Report /COW/NN_DDIC_CREATOR
*&---------------------------------------------------------------------*
*&
*&---------------------------------------------------------------------*
REPORT /cow/pal_ddic_creator.


DATA: ls_comp   TYPE /COW/PAL_s_COMP,
      lt_comp   TYPE /cow/pal_t_comp,
      lv_string TYPE string.

PARAMETERS: p_struc TYPE tabname DEFAULT 'ZZZ_S_TEST_1',
            p_ttyp  TYPE tabname DEFAULT 'ZZZ_T_TEST_1',
            p_dev   TYPE devclass DEFAULT '$TMP',
            p_descr TYPE text60 DEFAULT 'Description'.

TRY.
    ls_comp-name = 'ID'.
    ls_comp-ddic_type = '/COW/PAL_ID'.
    APPEND ls_comp TO lt_comp.

    DO 784 TIMES.
      lv_string = sy-index.
      ls_comp-name = 'X_' && |{ lv_string }|.
      ls_comp-ddic_type = '/COW/PAL_FLOAT'.
      APPEND ls_comp TO lt_comp.
    ENDDO.

    /cow/cl_pal_ddic_util=>static_create_ddic_struc(
        it_comp      = lt_comp
        iv_strucname = p_struc
        iv_tabname   = p_ttyp
        iv_descr     = p_descr
        iv_devclass  = p_dev ).
  CATCH /cow/cx_root INTO DATA(lo_exc).
    lo_exc->to_message( ).
ENDTRY.

The generation of the DDIC structure is done in the method /COW/CL_PAL_DDIC_UTIL =>STATIC_CREATE_DDIC_STRUCTURE:

class /COW/CL_PAL_DDIC_UTIL definition
  public
  final
  create public .

public section.

  class-methods STATIC_CREATE_DDIC_STRUC
    importing
      !IT_COMP type /COW/PAL_T_COMP
      !IV_STRUCNAME type TABNAME
      !IV_TABNAME type TABNAME
      !IV_DESCR type TEXT60
      !IV_DEVCLASS type DEVCLASS
    raising
      /COW/CX_ROOT .
protected section.
private section.
ENDCLASS.



CLASS /COW/CL_PAL_DDIC_UTIL IMPLEMENTATION.


* <SIGNATURE>---------------------------------------------------------------------------------------+
* | Static Public Method /COW/CL_PAL_DDIC_UTIL=>STATIC_CREATE_DDIC_STRUC
* +-------------------------------------------------------------------------------------------------+
* | [--->] IT_COMP                        TYPE        /COW/PAL_T_COMP
* | [--->] IV_STRUCNAME                   TYPE        TABNAME
* | [--->] IV_TABNAME                     TYPE        TABNAME
* | [--->] IV_DESCR                       TYPE        TEXT60
* | [--->] IV_DEVCLASS                    TYPE        DEVCLASS
* | [!CX!] /COW/CX_ROOT
* +--------------------------------------------------------------------------------------</SIGNATURE>
  METHOD static_create_ddic_struc.

    DATA: ls_dd02v   TYPE dd02v,
          ls_dd09v   TYPE dd09v,
          lt_dd03p   TYPE TABLE OF dd03p,
          ls_dd03p   TYPE  dd03p,
          ls_message TYPE bapiret2,
          ls_dd40v   TYPE dd40v.


*   1.) Create structure
*   Fill header
    ls_dd02v-tabname = iv_strucname.
    ls_dd02v-ddlanguage = sy-langu.
    ls_dd02v-tabclass = 'INTTAB'.
    ls_dd02v-exclass  = '3'.
    ls_dd02v-ddtext = iv_descr.
    ls_dd02v-as4user = sy-uname.
    ls_dd02v-as4date = sy-datum.
    ls_dd02v-as4time = sy-uzeit.
    ls_dd02v-exclass = '1'.

*   Fields
    LOOP AT it_comp ASSIGNING FIELD-SYMBOL(<ls_comp>).
      DATA(lv_tabix) = sy-tabix.
      ls_dd03p-tabname = iv_strucname.
      ls_dd03p-fieldname = <ls_comp>-name.
      ls_dd03p-fieldname = ls_dd03p-fieldname.
      TRANSLATE ls_dd03p-fieldname TO UPPER CASE.
      ls_dd03p-rollname = <ls_comp>-ddic_type.
      ls_dd03p-ddlanguage = 'D'.
      ls_dd03p-position = lv_tabix.
      APPEND ls_dd03p TO lt_dd03p.
    ENDLOOP.

    CALL FUNCTION 'DDIF_TABL_PUT'
      EXPORTING
*       CTRL_TABL_PUT       = 'XXXXXX'
        dd02v_wa            = ls_dd02v
        name                = iv_strucname
      TABLES
        dd03p_tab           = lt_dd03p
      EXCEPTIONS
        db_access_failure   = 1
        object_inconsistent = 2
        OTHERS              = 3.
    IF sy-subrc <> 0.
      ls_message-id = '/COW/MESS_PAL'.
      ls_message-number = '000'.
      ls_message-type = 'E'.
      ls_message-message_v1 = iv_tabname.
      RAISE EXCEPTION TYPE /cow/cx_root
        EXPORTING
          gs_message = ls_message.
    ENDIF.

    CALL FUNCTION 'DDIF_TABL_ACTIVATE'
      EXPORTING
        name        = iv_strucname
        excommit    = abap_false
      EXCEPTIONS
        not_found   = 1
        put_failure = 2
        OTHERS      = 3.


    DATA: lv_objname   TYPE tadir-obj_name,
          lv_srcsystem TYPE tadir-srcsystem.

    lv_objname = iv_strucname.
    lv_srcsystem = sy-sysid.

*   TADIR
    CALL FUNCTION 'TR_TADIR_INTERFACE'
      EXPORTING
        wi_test_modus      = space
        wi_tadir_pgmid     = 'R3TR'
        wi_tadir_object    = 'TABL'
        wi_tadir_obj_name  = lv_objname
        wi_tadir_srcsystem = lv_srcsystem
        wi_tadir_author    = sy-uname
        wi_tadir_devclass  = iv_devclass
        wi_set_genflag     = 'X'
      EXCEPTIONS
        OTHERS             = 25.

*   2.) Create table type
    ls_dd40v-typename   = iv_tabname.
    ls_dd40v-rowtype    = iv_strucname.
    ls_dd40v-rowkind    = 'S'.
    ls_dd40v-accessmode = 'T'.
    ls_dd40v-keydef     = 'D'.
    ls_dd40v-keykind    = 'G'.
    ls_dd40v-ddlanguage = sy-langu.
    ls_dd40v-ddtext     = iv_descr.

    CALL FUNCTION 'DDIF_TTYP_PUT'
      EXPORTING
        name              = iv_tabname
        dd40v_wa          = ls_dd40v
      EXCEPTIONS
        ttyp_not_found    = 1
        name_inconsistent = 2
        ttyp_inconsistent = 3
        put_failure       = 4
        put_refused       = 5
        OTHERS            = 6.
    IF sy-subrc <> 0.
      ls_message-id = '/COW/MESS_PAL'.
      ls_message-number = '001'.
      ls_message-type = 'E'.
      ls_message-message_v1 = iv_tabname.
      RAISE EXCEPTION TYPE /cow/cx_root
        EXPORTING
          gs_message = ls_message.
    ENDIF.

    CALL FUNCTION 'DDIF_TTYP_ACTIVATE'
      EXPORTING
        name        = iv_tabname
      EXCEPTIONS
        not_found   = 1
        put_failure = 2
        OTHERS      = 3.

    lv_objname = iv_tabname.
    lv_srcsystem = sy-sysid.
*   TADIR
    CALL FUNCTION 'TR_TADIR_INTERFACE'
      EXPORTING
        wi_test_modus      = space
        wi_tadir_pgmid     = 'R3TR'
        wi_tadir_object    = 'TTYP'
        wi_tadir_obj_name  = lv_objname
        wi_tadir_srcsystem = lv_srcsystem
        wi_tadir_author    = sy-uname
        wi_tadir_devclass  = iv_devclass
        wi_set_genflag     = 'X'
      EXCEPTIONS
        OTHERS             = 25.

  ENDMETHOD.
ENDCLASS.

The result:

This code can easily be re-used by substituting the namespace /COW/ by a customer namespace. Of course the program can easily be adapted to other dataframe requirements.