JSON or XML Input Using the Data-Into Parser

Owned by Former user (Deleted)
Last updated: Aug 16, 2023 by Jessica Tomkins
10 min read



The Profound UI Universal Display File Data-Into Parser (PUIUDFINTO) lets you receive JSON or XML documents into web services running in the Universal Display File environment.  These documents are automatically fed into RPG's DATA-INTO opcode and loaded into a matching data structure.

Requirements:

  • ILE RPG compiler for IBM i 7.2 or newer.

  • Profound UI version 6, fix pack 1.0 or newer.

  • Users running IBM i 7.2 or 7.3 will need PTFs that enable DATA-INTO

Description:

DATA-INTO is a feature of IBM's ILE RPG that lets you use a 3rd-party "parser" to interpret a document, and have its output automatically load into RPG variables.  Profound UI provides a parser called PUIUDFINTO that will read a document that is sent to the Universal Display File interface, and if it is an XML or JSON document, it will parse it and return its values through the DATA-INTO interface. A full explanation of all of DATA-INTO and all of its options is available in the ILE RPG Reference Manual provided by IBM.  This document will only provide a brief overview of its functionality, but will explain the details that are specific to the PUIUDFINTO parser.

If you are familiar with RPG's XML-INTO opcode, then you will find DATA-INTO to be very similar.  The primary difference is that XML-INTO only understands XML, whereas DATA-INTO can understand any document format, provided that you can find a parser program for that format. Profound's PUIUDFINTO program understands both XML and JSON so can be used with either format.

Introduction to DATA-INTO:

If you're not familiar with DATA-INTO or XML-INTO, the easiest way to understand is to look at an example.  Consider this JSON document:

{ "custno": 1000, "name": "ACME, Inc." }

In JSON, the curly braces mean that it is an object, which is the same as what RPG calls a data structure. This object has two sub-fields, named "custno" and "name".  The colon character separates the field name from its value, so you could say this object has a field named "custno" that is set to 1000 and a field named "name" that is set to "ACME, Inc."  It is identical to the following definition in a traditional RPG program:

D Customer DS D custno 4p 0 inz(1000) D name 30a inz('ACME, Inc')

Or in newer programs, you may choose to use a free-format RPG data structure to represent the same thing.

dcl-ds Customer; custno packed(4: 0) inz(1000); name char(30) inz('ACME, Inc'); end-ds;

The purpose of DATA-INTO is simply to take the JSON document (or another document such as XML) and load it into a corresponding RPG variable.  For DATA-INTO to work, the RPG variable must be the same format as the document you are loading.  In this case, since the JSON document is an object, the RPG variable must be a data structure, and since the JSON document has fields named 'custno' and 'name', the RPG data structure must also have fields named 'custno' and 'name'.  The RPG syntax to load this particular JSON document using PUIUDFINTO would look like this:

This tells DATA-INTO to map the data from the JSON document into the 'Customer' data structure. 

The %DATA built-in function is where you tell RPG about the document you're parsing.  Normally, the first parameter to %DATA is a string containing the document to be parsed, however, the PUIUDFINTO parser does not use this parameter because it reads the JSON or XML document from the network connection. For that reason, this example passes an empty string as the first parameter to %DATA.  The second parameter is where you provide options provided by IBM that control how DATA-INTO processes the data. In this example, I've used the 'case=convert' option, which will allow the field names in the JSON document to be converted to valid RPG field names.

PUIUDFINTO supports all of the options that IBM provides for %DATA except for the "doc=file" option.  Since it always reads the document from the network, doc=file will not work.  For this reason, we recommend that you never code the doc option.

The %PARSER built-in function tells DATA-INTO which program to use as it's 3rd-party parser.  PUIUDFINTO is the name of the *PGM object that Profound UI provides for Universal Display Files. Since we did not specify a library name, it will use the library list to find PUIUDFINTO, for that reason you'll need the Profound UI library in your library list. The Universal Display File Web Connector will place the Profound UI library in your library list automatically.

After running the DATA-INTO opcode as shown in the preceding example, the 'Customer' data structure will contain the values from the JSON document.

Using an XML Document

The PUIUDFINTO program works with both JSON and XML documents, it will automatically determine the type of the document that was sent to the Universal Display File, and interpret it as XML or JSON as appropriate.

For example, in the "Introduction to DATA-INTO" section, above, I could replace the JSON document with the following XML document:

Like the JSON document, this XML document would be mapped into the RPG data structure.  The RPG code to do this is identical to the RPG code shown above in the JSON example. The 'custno' tag would be loaded into the RPG 'custno' field, and the 'name' tag would be loaded into the RPG 'name' field.  The difference between JSON and XML is transparent to the RPG program.

Parser Options

The %PARSER built-in function has a second parameter where options may be specified.  PUIUDFINTO expects these options to always be passed in the form of a character string containing a JSON object.  The fields in the JSON document are all optional, you only need to specify the ones that are needed.

The options are as follows:

  • value_null = (JSON only) character value to be placed into an RPG field when a JSON document specifies that a field is null. (Default: '*NULL')

  • value_true = (JSON only) character value to be placed into an RPG field when a JSON document specifies that a field is set to boolean true.  (Default: '1')

  • value_false = (JSON only) character value to be placed into an RPG field when a JSON document specifies that a field is set to boolean false. (Default: '0')

  • document_name = (JSON only) the name of the outermost (aka "document-level") JSON node.  (Default: unspecified)

  • datasubf = (XML only) subfield to load tag data, similar to the RPG XML-INTO 'datasubf' option.  (Default: 'value')

  • ns = (XML only) namespace option. "remove" specifies that namespaces are removed from the XML element name before it is converted to an RPG field name.  "keep" specifies that they are kept.  (Default: 'keep')

  • nsprefix = (XML only) namespace prefix.  When the ns option is set to "remove", this option can be used to specify a field that the namespace will be copied into.  For example, if the nsprefix is set to "ns_" and you are stripping the namespace from a field name "name", the contents of the namespace will be loaded into an RPG field named "ns_name".  (Default: removed namespaces are not mapped into fields)

  • content_type_field = A field at the top-level of the RPG data structure that is to contain the content-type from the HTTP transaction.  (Default: the content type is not mapped)

  • request_method_field = A field at the top-level of the RPG data structure that is to contain the request method from the HTTP transaction.  (Default: the request method is not mapped)

In the preceding list, when an option says "JSON only" it will be ignored unless your web service is consuming a JSON document.  Likewise, if it says "XML only", the option is ignored unless it is consuming an XML document.

Example:

In this example, if the caller sends a JSON document, boolean true and false values will be given character strings of "true" and "false", respectively.  If the caller sends an XML document, any namespaces will be removed from the element names before mapping them into RPG fields.  The other options that were not specified, will all use their default settings.

Useful Environment Variables

Since PUIUDFINTO is used from the Universal Display File environment, any of the environment variables available in the IBM HTTP Server (powered by Apache) are available for use in your web service.  Explaining all of them is beyond the scope of this document, however here is a short list of variables that are very common in web service programs:

  • REQUEST_METHOD = The HTTP method that was used to access this service. Typical values are GET, POST, PUT or DELETE.

  • CONTENT_TYPE = The internet media type (often called "mime type") of the document that was sent, such as "application/xml" or "application/json"

  • HTTP_ACCEPT = A list of media types the caller has indicated that they will accept.  Often used to determine the type of data to send back.

  • PUI_UNIVERSAL_URI = The URI (often called 'URL") that was used to run this service.  Typically it will be something like "http://your-system/profoundui/universal/MYPGM" and can have additional variables or parameters appended to it.

You can retrieve environment variables using the IBM-supplied getenv() API.  Examples of using the getenv() API can be found under HTTP Input and Field Validation.

Customer Display/Update Example

This provides an example of creating a web service that can be used to retrieve or update a customer's address. Input to the web service is provided in the form of either a JSON or XML document that is processed using PUIUDFINTO.  Output is provided in a similar format, created with a Universal Display File.

Source code:

  • CUSTADRP.SQL = SQL Script to create the CUSTADRP database table and populate it with a few sample customers.

  • CUSTSERVD.DSPF = Universal Display File to be used by the CUSTSERVR RPG program

  • CUSTSERVR.SQLRPGLE = ILE RPG (with SQL) source for the CUSTSERVR program

  • CUSTSERVCL.CLLE = ILE CL source to set the library list for the CUSTSERVR program

To build and install the example:

  • Upload the members by copy/pasting them into RDi, or alternately, upload them with FTP in ASCII mode.

  • These instructions assume that source files named QSQLSRC, QDDSSRC, QRPGLESRC and QCLSRC are used for the SQL, DSPF, SQLRPGLE and CLLE source, respectively. If you chose a different source file, please substitute it in the following instructions

  • Edit QCLSRC,CUSTSERVCL and change the ADDLIBLE commands to point to your own library.

  • CHGCURLIB CURLIB(your-library) to set the library where the file and source members will be compiled

  • RUNSQLSTM SRCFILE(QSQLSRC) SRCMBR(CUSTADRP) to create the database table

  • CRTDSPF FILE(CUSTSERVD) SRCFILE(QDDSSRC) to create the Universal Display file

  • CRTSQLRPGI OBJ(CUSTSERVR) DBGVIEW(*SOURCE) SRCFILE(QRPGLESRC) to create the RPG program

  • CRTBNDCL PGM(CUSTSERVCL) SRCFILE(QCLSRC) to create the CL program

  • Use the PUIWRKMAP command to add your mapping for the Universal Display File. Click "Add" and use these values:

    • URI MAPPING: /custserv

    • LIBRARY: your-library

    • PROGRAM: CUSTSERVCL

    • SIGNON: No

    If you cannot use PUIWRKMAP, you can add these to the PUIMAPP table yourself. Details can be found under Using the Web Connector.

Trying the Example

This example program is a web service provider (or "API") only. Writing a program to consume this service is left as an exercise for the reader. An easy way to test it would be to use a web service testing tool such as SoapUI or PostMan.

Retrieve Example:

  • Make a GET request to http://your-system:port/profoundui/universal/custserv/1234

  • Try adding an 'Accept' header set to 'application/xml' or 'application/json' to test getting a response in XML or JSON format.

  • The above example URI ends in 1234, which is the customer number.  Try using 1000 or 2000 to retrieve different customers.

Update Example:

  • Try a PUT request to update the customer's address.  You will need to send a JSON or XML document in the request body. Depending on the tool you're using, it may be necessary to tell the tool that the data is "application/json" or "application/xml" (text/xml will also work for XML) so that it understands the format of the data you're sending.


    JSON format:

    XML format:



  • You can change the name or address in your XML or JSON document to update the database. The API should return a document in the same format to show you the updated data.