Creating a JSON Parsing Subprocedure
The RPG API Express subprocedure RXS_ParseJson() is used to read through a JSON document and trigger processing based on JSON events detected in that document. These events represent objects, arrays, and fields in the JSON document and can used to tailor processing of the JSON document to meet your business needs through a customized parsing handler subprocedure. This parsing handler subprocedure is defined within your program, and is specified in the ParseJsonDS.Handler field on the RXS_ParseJson() configuration data structure parameter.
Depending on the size and complexity of your JSON document, there may be many individual events that you need to capture to properly retrieve and handle your data. To help simplify this process, the command BLDPRS can be used to generate a basic parsing handler for your JSON document, into which you can add your custom programming and data handling.
JSON Parsing Events
The following element event types within a JSON document will trigger a call out to the parsing handler subprocedure:
| Event | pType Constant | pPath Format | Example | |||
|---|---|---|---|---|---|---|
| Array Start | RXS_JSON_ARRAY |
[*] |
/phone[*] |
|||
| Array End | RXS_JSON_ARRAY_END |
[*] |
/phone[*] |
|||
| Object Start | RXS_JSON_OBJECT |
/ |
/phone |
|||
| Object End | RXS_JSON_OBJECT_END |
/ |
/phone |
These events are triggered by objects and arrays in the JSON document. These events will not contain data to be retrieved - instead, these events are most often used to control program flow by initializing values or writing records out to physical files.
Additionally, there are multiple types of content events that can be triggered by data found within the JSON document:
| Data Type | pType Constant | RPG Data Type | ||
|---|---|---|---|---|
| String | RXS_JSON_STRING |
A |
||
| Boolean | RXS_JSON_BOOLEAN |
N |
||
| Integer | RXS_JSON_INTEGER |
20I 0 |
||
| Double | RXS_JSON_DOUBLE |
8F 0 |
||
| Null | RXS_JSON_NULL |
The default behavior of RXS_ParseJson() is that all content, regardless of data type, will be returned as an RXS_JSON_STRING event type and as RPG character data. This is the recommended approach to parsing, especially for documents that contain decimal or currency data. This behavior can be controlled with the ParseJsonDS.ConvertDataToString subfield in the configuration data structure when calling RXS_ParseJson().
Sample JSON Document
In order to generate a parsing handler, you will first need a sample JSON document that is as complete as possible - that is, you should include even objects and fields and which are optional, even if some of them would not logically appear with others in normal usage.
To start, we first need to create a file in the IFS. This file will be populated with our sample JSON data, which will be used to generate the parser. The command below uses QSHELL to create an IFS stream file in the /tmp directory using CCSID 819. Note that this command is case-sensitive.
QSH CMD('touch -C 819 /tmp/leads.json')
The file now exists but without content. To add content you can use the EDTF command, or you can edit the file using Rational Developer for i or a number of other IBM i tools.
EDTF '/tmp/leads.json'
For demonstration purposes, we’ll use the sample JSON below.
{
"leads": [
{
"name": "Dona Franks",
"salesProspect": true,
"address": {
"streetNumber": 20391,
"apartment": "177",
"street": "Central Avenue",
"city": "Waikele",
"state": "MN",
"postalCode": "60247"
},
"phone": [
"+1 (971) 596-2501",
"+1 (948) 493-2985"
],
"email": "donafranks@example.com"
},
{
"name": "Ortega Stuart",
"salesProspect": false,
"address": {
"streetNumber": 76308,
"apartment": null,
"street": "Delmonico Place",
"city": "Fillmore",
"state": "VA",
"postalCode": "94862"
},
"phone": [
"+1 (977) 470-3280"
],
"email": "ortegastuart@example.com"
}
]
}
While in the Edit File editor, select F2 to save the document changes. Now it is time to call the BLDPRS command to generate the parsing code.
Generating the Parsing Handler (BLDPRS)
BLDPRS can generate parsing handlers for both RXS 2 and RXS 3 XML parsers as well as the RXS 3 JSON parser. The command can also generate either free- or mixed-format (fixed D-specs) RPG code, and this code can be output into either a source member or an IFS file. Prompting the BLDPRS command will present the following screen:
The first parameter will be the fully-qualified filepath for our sample JSON document in the IFS. For demonstration purposes, we’re going to output the generated parsing handler code to a source member, which means we’ll be specifying the Output Source File, Library, and member fields. When writing output to a source member, we do not specify a value in the Output Stream File field.
The Append Output parameter controls whether any existing content in the specified output member or stream file will be preserved, or whether it will be overwritten. By default, this value is set to *YES to preserve existing content. In our demonstration, we’ll set it to *NO to overwrite any content.
The final parameter - Parsing Handler Type - has three possible values and controls what type of parsing handler is generated. We’ll input *JSON to indicate that a JSON parsing handler, for use with RXS_ParseJson(), should be generated.
The Parsing Handler Subprocedure
Once the command finishes executing, you’ll see a status message similar to the following:
Generated parsing handler in USERLIB/QRPGLESRC, LEADPRS
If we open the generated source member in RDi or SEU, we’ll see this RPGLE code:
Dcl-Pr JsonHandler Ind;
pType Int(5) Const;
pPath Like(RXS_Var64Kv_t) Const;
pIndex Uns(10) Const;
pData Pointer Const;
pDataLen Uns(10) Const;
End-Pr;
//=======================================================================
// Remember to update the handler to account for your program logic.
//=======================================================================
Dcl-Proc JsonHandler;
Dcl-Pi *N Ind;
pType Int(5) Const;
pPath Like(RXS_Var64Kv_t) Const;
pIndex Uns(10) Const;
pData Pointer Const;
pDataLen Uns(10) Const;
End-Pi;
Dcl-S ParsedData Like(RXS_Var1Kv_t);
select;
when pPath = '/'
and pType = RXS_JSON_OBJECT;
RXS_JobLog( '***Object Start: ' + pPath );
when pPath = '/leads[*]'
and pType = RXS_JSON_ARRAY;
RXS_JobLog( '***Array Start: ' + pPath );
when pPath = '/leads[*]'
and pType = RXS_JSON_OBJECT;
RXS_JobLog( '***Object Start: ' + pPath );
when pPath = '/leads[*]/name'
and pType = RXS_JSON_STRING;
ParsedData = RXS_STR( pData : pDataLen );
RXS_JobLog( pPath + ': ' + ParsedData );
when pPath = '/leads[*]/salesProspect'
and pType = RXS_JSON_STRING;
ParsedData = RXS_STR( pData : pDataLen );
RXS_JobLog( pPath + ': ' + ParsedData );
when pPath = '/leads[*]/address'
and pType = RXS_JSON_OBJECT;
RXS_JobLog( '***Object Start: ' + pPath );
when pPath = '/leads[*]/address/streetNumber'
and pType = RXS_JSON_STRING;
ParsedData = RXS_STR( pData : pDataLen );
RXS_JobLog( pPath + ': ' + ParsedData );
when pPath = '/leads[*]/address/apartment'
and pType = RXS_JSON_STRING;
ParsedData = RXS_STR( pData : pDataLen );
RXS_JobLog( pPath + ': ' + ParsedData );
when pPath = '/leads[*]/address/street'
and pType = RXS_JSON_STRING;
ParsedData = RXS_STR( pData : pDataLen );
RXS_JobLog( pPath + ': ' + ParsedData );
when pPath = '/leads[*]/address/city'
and pType = RXS_JSON_STRING;
ParsedData = RXS_STR( pData : pDataLen );
RXS_JobLog( pPath + ': ' + ParsedData );
when pPath = '/leads[*]/address/state'
and pType = RXS_JSON_STRING;
ParsedData = RXS_STR( pData : pDataLen );
RXS_JobLog( pPath + ': ' + ParsedData );
when pPath = '/leads[*]/address/postalCode'
and pType = RXS_JSON_STRING;
ParsedData = RXS_STR( pData : pDataLen );
RXS_JobLog( pPath + ': ' + ParsedData );
when pPath = '/leads[*]/address'
and pType = RXS_JSON_OBJECT_END;
RXS_JobLog( '***Object End: ' + pPath );
when pPath = '/leads[*]/phone[*]'
and pType = RXS_JSON_ARRAY;
RXS_JobLog( '***Array Start: ' + pPath );
when pPath = '/leads[*]/phone[*]'
and pType = RXS_JSON_STRING;
ParsedData = RXS_STR( pData : pDataLen );
RXS_JobLog( pPath + ': ' + ParsedData );
when pPath = '/leads[*]/phone[*]'
and pType = RXS_JSON_ARRAY_END;
RXS_JobLog( '***Array End: ' + pPath );
when pPath = '/leads[*]/email'
and pType = RXS_JSON_STRING;
ParsedData = RXS_STR( pData : pDataLen );
RXS_JobLog( pPath + ': ' + ParsedData );
when pPath = '/leads[*]'
and pType = RXS_JSON_OBJECT_END;
RXS_JobLog( '***Object End: ' + pPath );
when pPath = '/leads[*]'
and pType = RXS_JSON_ARRAY_END;
RXS_JobLog( '***Array End: ' + pPath );
when pPath = '/'
and pType = RXS_JSON_OBJECT_END;
RXS_JobLog( '***Object End: ' + pPath );
endsl;
return RXS_JSON_CONTINUE_PARSING;
End-Proc;
Note that, even with the Code Format parameter set to *FREE, the generated free-format RPG code is still limited to columns 8-80. This is for compatibility with customers that are using SEU for development, or for customers on 7.1 that do not have the PTFs required for fully free-format RPG code. Though our example JSON document does not have any, long paths will wrap automatically at or before the 80th column.
The generated code member contains the parsing handler subprocedure prototype, which may be copied into your program code in the main D-specs (though this is not generally required anymore), and the parsing subprocedure code itself. You should not modify the prototype or the subprocedure declarations, unless it is to change the subprocedure name from JSONHandler.
The body of the parsing handler subprocedure contains a select block, composed of when statements for each event in the JSON document. These when statements are checking the value of the pPath parameter to determine what event was detected, and processing that event accordingly.
For example, the below when statement is checking if the pPath parameter contains the path to the name attribute within the leads object, and that the pType is RXS_JSON_STRING. pPath can be identical in parts of the JSON document, so checking pType helps us ensure that this element has character data we can then retrieve with RXS_STR():
when pPath = '/leads[*]/name'
and pType = RXS_JSON_STRING;
ParsedData = RXS_STR( pData : pDataLen );
For each of these statements, the generated code also includes a call to RXS_JobLog(). This is intended to be for demonstrative purposes, so that you can copy the parsing handler into your program and immediately compile and run the program to see sample output in the job log. These RXS_JobLog calls should be removed and replaced by your actual program logic to handle the data in the JSON document.
Customizing the Parsing Handler Subprocedure
Once the generated parsing handler subprocedure code has been copied into your program, you will need to modify the programming logic in the handler subprocedure to meet the needs of your specific program. This may involve retrieving data from attributes, performing numeric or data-type conversions, or writing records to one or more physical files. You may also need to store the data within the program for further processing once parsing is complete. The parsing handler subprocedures are very flexible and allow for a high level of customization.
The generated parsing handler subprocedure will contain a when statement for each JSON event detected in the document. In practice, you will likely not need most of these events for processing. You can safely comment out or remove any unneeded events.
Sample Program
Here is a full example that demonstrates parsing our sample JSON using a fully-customized parsing handler subprocedure, based on the same code we generated in this tutorial:
Ctl-Opt ActGrp(*New) BndDir('RXSBND') Text('RXS JSON Handler Example');
/COPY QRPGLECPY,RXSCB
// Global data structure we're going to use
// to store our parsed JSON data
Dcl-Ds Leads Qualified Dim(5) Inz;
Name VarChar(30);
SalesProspect Ind;
AddressNumber Int(10);
Apartment VarChar(30);
Street VarChar(50);
City VarChar(50);
State Char(2);
PostalCode VarChar(10);
PhoneNumbers VarChar(20) Dim(5);
Email VarChar(256);
End-Ds;
// Global field we'll use to count how many total leads
// we parsed from our JSON.
Dcl-S LeadCount Uns(5);
// Used when we're looping through our Leads data structure
// to output parsed data.
Dcl-S i Uns(5);
// RXS templated data structures
Dcl-Ds ParseJsonDS LikeDS(RXS_ParseJsonDS_t);
// Many RPG API Express APIs accept a configuration data structure
// parameter. These templated data structures must be initialized
// using RXS_ResetDS - they cannot be initialized with the reset
// operation.
// RXS_ResetDS helps ensure that RXS templated data structures are
// initialized in a backwards-compatible fashion
RXS_ResetDS( ParseJsonDS : RXS_DS_TYPE_PARSEJSON );
// For demonstration purposes, we're going to load the JSON from the
// /tmp/leads.json file created in the IFS earlier in this process.
ParseJsonDS.Stmf = '/tmp/leads.json';
// Specify the procedure pointer for the parsing handler subprocedure,
// using the %PAddr built-in function
ParseJsonDS.Handler = %PAddr( JSONHandler );
monitor;
// Parse the JSON document. Because we're loading the JSON from an
// IFS file we can *Omit the first parameter - otherwise, that's
// where we would pass JSON in as a field.
RXS_ParseJson( *Omit : ParseJsonDS );
// We're just going to output the name and email for each lead
// to demonstrate that we parsed the data.
for i = 1 to LeadCount;
RXS_JobLog( 'Lead #: %s' : %Char(i) );
RXS_JobLog( 'Name: %s' : Leads(i).Name );
RXS_JobLog( 'Email: %s' : Leads(i).Email );
endfor;
on-error;
// If an error occurs during parsing, error messages and information
// can be found in the ParseDS parameter data structure
RXS_JobLog( 'Error: ' + ParseJsonDS.ReturnedErrorInfo.MessageText );
endmon;
*INLR = *On;
return;
// This is a customized parsing handler subprocedure that was initially
// generated with BLDPRS. Additional code has been added to support our
// program logic, and we've removed the when blocks for XPaths that
// we do not need to process.
// We have also removed the calls to RXS_JobLog - we do not recommend
// leaving logging operations in place in a production environment, due
// to the additional overhead
Dcl-Proc JsonHandler;
Dcl-Pi *N Ind;
pType Int(5) Const;
pPath Like(RXS_Var64Kv_t) Const;
pIndex Uns(10) Const;
pData Pointer Const;
pDataLen Uns(10) Const;
End-Pi;
Dcl-S ParsedData Like(RXS_Var1Kv_t);
select;
// This will detect the root object for our JSON document.
// Typically an event like this may be used to reset
// data structures or open files.
when pPath = '/'
and pType = RXS_JSON_OBJECT;
// We'll initialize our Leads data structure here.
reset Leads;
// This is detecting the start of our 'leads' array. Array
// start events are also a good place to perform resets
// or file opens, typically for data structures or files
// closely related to the array being handled.
when pPath = '/leads[*]'
and pType = RXS_JSON_ARRAY;
// We'll initialize our LeadCount field here.
reset LeadCount;
// Note that this has the same '/leads[*]' pPath as the
// above event, but this has a pType of RXS_JSON_OBJECT
// instead of RXS_JSON_ARRAY. This is because JSON arrays
// do not contain key/value pairs like JSON objects do,
// and instead just contain a list of values without names.
// Because each object within our 'leads' array doesn't
// have a name, there's no way to make pPath more unique,
// so pType lets us make the distinction between the array
// and the object.
when pPath = '/leads[*]'
and pType = RXS_JSON_OBJECT;
// We're keeping a count of how many total leads we parsed,
// so we're going to use this event to increment our
// LeadCount field.
LeadCount += 1;
// As we've defined it, our Leads data structure array can
// only hold so many records - if we receive more, we're
// going to print a message to the job log and stop
// parsing.
if LeadCount > %Elem(Leads);
RXS_JobLog( 'Too many leads to store!' );
// This is explained further at the end of this subproc.
return RXS_JSON_STOP_PARSING;
endif;
when pPath = '/leads[*]/name'
and pType = RXS_JSON_STRING;
Leads(LeadCount).Name = RXS_STR( pData : pDataLen );
when pPath = '/leads[*]/salesProspect'
and pType = RXS_JSON_STRING;
monitor;
// We're getting 'true' or 'false' back as a string
// so we need to do some logic to convert that to
// an indicator. We'll first retrieve the data
// into ParsedData as a work field.
ParsedData = RXS_STR( pData : pDataLen );
if ParsedData = 'true';
Leads(LeadCount).SalesProspect = RXS_YES;
else;
Leads(LeadCount).SalesProspect = RXS_NO;
endif;
on-error;
RXS_JobLog( 'Failed to convert salesProspect!' );
endmon;
// In this case, the 'address' object was just used
// to group a number of related properties together
// in the JSON document. We don't really need to
// do anything with the object start or end events
// as a result.
when pPath = '/leads[*]/address'
and pType = RXS_JSON_OBJECT;
// We don't have to do anything here in this example. This
// 'when' statement could be removed from this program.
when pPath = '/leads[*]/address/streetNumber'
and pType = RXS_JSON_STRING;
monitor;
// This is a numeric JSON field, specifically an
// integer, so we need to do special handling to
// convert from string to int.
ParsedData = RXS_STR( pData : pDataLen );
Leads(LeadCount).StreetNumber = %Uns( ParsedData );
on-error;
RXS_JobLog( 'Failed to convert streetNumber!' );
endmon;
when pPath = '/leads[*]/address/apartment'
and pType = RXS_JSON_STRING;
Leads(LeadCount).Apartment = RXS_STR( pData : pDataLen );
when pPath = '/leads[*]/address/street'
and pType = RXS_JSON_STRING;
Leads(LeadCount).Street = RXS_STR( pData : pDataLen );
when pPath = '/leads[*]/address/city'
and pType = RXS_JSON_STRING;
Leads(LeadCount).City = RXS_STR( pData : pDataLen );
when pPath = '/leads[*]/address/state'
and pType = RXS_JSON_STRING;
Leads(LeadCount).State = RXS_STR( pData : pDataLen );
when pPath = '/leads[*]/address/postalCode'
and pType = RXS_JSON_STRING;
Leads(LeadCount).PostalCode = RXS_STR( pData : pDataLen );
// In this case, the 'address' object was just used
// to group a number of related properties together
// in the JSON document. We don't really need to
// do anything with the object start or end events
// as a result.
when pPath = '/leads[*]/address'
and pType = RXS_JSON_OBJECT_END;
// We don't have to do anything here in this example. This
// 'when' statement could be removed from this program.
when pPath = '/leads[*]/phone[*]'
and pType = RXS_JSON_ARRAY;
// We don't have to do anything here in this example. This
// 'when' statement could be removed from this program.
when pPath = '/leads[*]/phone[*]'
and pType = RXS_JSON_STRING;
// We haven't used the pIndex parameter yet, but it's
// going to be helpful here. pIndex contains the
// index of our current JSON property within the
// parent object or array starting from 1. So for this
// 'phones' array, the first value in 'phones' would have
// a pIndex of 1, the second would have a pIndex of 2,
// and so on. In this case we just have to be careful
// that we aren't trying to store more phone numbers than
// our data structure has space for.
if pIndex <= %Elem(Leads(LeadCount).PhoneNumbers);
Leads(LeadCount).PhoneNumbers(pIndex) =
RXS_STR( pData : pDataLen );
else;
RXS_JobLog( 'Too many phone numbers for this lead!' );
endif;
when pPath = '/leads[*]/phone[*]'
and pType = RXS_JSON_ARRAY_END;
// We don't have to do anything here in this example. This
// 'when' statement could be removed from this program.
when pPath = '/leads[*]/email'
and pType = RXS_JSON_STRING;
Leads(LeadCount).Email = RXS_STR( pData : pDataLen );
when pPath = '/leads[*]'
and pType = RXS_JSON_OBJECT_END;
// We don't have to do anything here in this example. This
// 'when' statement could be removed from this program.
// This is detecting the end of our 'leads' array. Array
// end events are typically a good place to close files.
when pPath = '/leads[*]'
and pType = RXS_JSON_ARRAY_END;
// We don't have to do anything here in this example. This
// 'when' statement could be removed from this program.
// This will detect the end of the root object for our
// JSON document. Typically an event like this may be
// used to close files.
when pPath = '/'
and pType = RXS_JSON_OBJECT_END;
// We don't have to do anything here in this example. This
// 'when' statement could be removed from this program.
endsl;
// A parsing handler must always return RXS_JSON_CONTINUE_PARSING
// or RXS_JSON_STOP_PARSING. RXS_JSON_CONTINUE_PARSING will cause
// the parser to continue to the next event within the JSON
// document being parsed, while RXS_JSON_STOP_PARSING will cause
// the parser to stop parsing the document. This is useful when
// performing error handling as well as if you know you only data
// from a specific portion of a JSON document. By returning
// RXS_JSON_STOP_PARSING you can ignore the rest of the JSON
// document, helping your program run more efficiently.
return RXS_JSON_CONTINUE_PARSING;
End-Proc;