Print

JSON (JavaScript Object Notation) is fast becoming an essential technology in modern IBM i shops. From web services to browser interfaces and data exchange—it has seen a remarkable growth in usage over the last few years. In recent times IBM have even added direct JSON support to DB2 - a topic we may return to another day.

JSON began basically as a replacement for XML in Ajax calls. That’s somewhat ironic when you consider that the "x" in Ajax actually stands for XML—“Asynchronous Javascript and XML."

XML had proven just too big and lumpy, requiring too much horsepower to parse in Web 2.0 type applications. JSON on the other hand is not only far more compact than XML, but because it is modelled on Javascript's own data definitions, a fast parser was already effectively built into every browser. So there was very little additional support needed in the browser. JSON also only uses UTF-8 encoding and so the multiple data encodings supported by XML do not have to be dealt with. With this three-fold advantage (compactness, ease of parsing, everything in UTF-8) it was hardly surprising that JSON rapidly replaced XML in most Ajax calls. From there it has gone on to become the vehicle of choice for web service requests and responses, and is rapidly entering the realm of data interchange.

JSON Syntax Basics

So, let's begin this brief introduction by looking at the basic rules for formatting JSON. JSON data consists of a series of name/value pairs with colons acting as the separator between the name and the value. Individual elements within a series are separated by commas and names must always be in quotes. So JSON data has this basic form:

	"name1" : value_1, "name2" : value_2, ... "namen" : value_n

The actual value portion of the name/value pair can be more than just a simple number or string. In fact it can be any of the following:

Number (integer or floating point)
String (enclosed in quotes)
Array
Object
Boolean true/false
or null.

Arrays are delineated by square brackets ( "[" and "]" ). Unlike RPG arrays, which are limited to containing multiple elements of the same type and size, JSON arrays can contain any type of value, including another array. Which in turn can contain an array, which can ... You get the picture.

Objects are delineated by curly braces ( "{" and "}" ) and can contain any other type of value including objects. Don't let the term "Object" confuse you - there is nothing OO about them. They are basically just RPG Data Structures (DS).

That's really all there is to it, other than to mention that if a string value contains a control character (of which quotes will probably be the most common) then it must be escaped by using the backslash ( "\" ) character in front of it. So to include a quote in a string you would code: "This value includes a \"quote\"" and similarly the "\" character would have to appear as "\\" in the string.

We may have over simplified things a little but hopefully you'll understand the basics. Unlike XML which has its rules very well documented (some might well say over documented!) information on the syntax rules for JSON can be hard to come by as it appears to inevitably be bound up in comparisons with Javascript. That’s fine if you are fully conversant with Javascript but we are not, and suspect that that is also true for many other RPGers.

Now that we have a basic understanding of the JSON syntax, let's look at a very simple example:

(A) {
(B) "employees" : [
(C) { "firstName":"John", "lastName":"Doe" },
    { "firstName":"Anna", "lastName":"Smith" },
    { "firstName":"Peter", "lastName":"Jones" }
(D) ]}

A. We begin with "{" - i.e. the data is an object. This is also known as the document root, or root node.

B. The first field's name is "employees" and its value is an array "["

C. The elements of the array are in turn objects, each one containing the values for "firstName" and "lastName". The commas separating out the individual fields.

D. Finally the document wraps up by closing the array ( "]" ) and then the root ( " } " ).

If we were to represent this data in an RPG DS it would look like this:

dcl-ds employees  Dim(3)  Qualified;
   firstName  Char(n);
   lastname   Char(n);
end-ds;

Ignoring, of course, any logic needed to initialize the values of the fields in the DS array. Plus we have to specify the number of elements in an RPG array.

Since many of you may already be more familiar with XML, let's compare an XML document with its JSON equivalent to see what the differences are. Here's the XML document. As you can see the Customer detail repeats:

<Customers>
   <Customer Id="G011739">
      <Name>Langosh, Borer and Homenick</Name>
      <Address>
         <City>Danykafurt</City>
         <State>NV</State>
      </Address>
   </Customer>
   <Customer Id="E791196">
      <Name>Denesik, Kessler and Rolfson</Name>
....
Since we gave you an RPG comparison earlier, here's the equivalent Data Structure:

dcl-ds Customers;
         dcl-ds Customer Dim(3) Qualified;
            Id    Char(7);
            Name  Char(40);
            dcl-ds  Address;
              City   Char(30);
              State  Char(2);
            end-ds;
         end-ds;
       end-ds;

So what would the JSON equivalent look like? Here's one way to code it:

{
   "Customers": {
      "Customer": [
         {
            "Id": "G011739",
            "Name": "Langosh, Borer and Homenick",
            "Address": {
               "City": "Danykafurt",
               "State": "NV"
            }
         },
         {
            "Id": "E791196",
            "Name": "Denesik, Kessler and Rolfson",
        ...

Of course, just like XML, JSON is not dependent on whitespace and so the document would likely look far more like this in practice:

{ "Customers": { "Customer": [ { "Id": "G011739", "Name": 
"Langosh, Borer and Homenick", "Address": { "City": "Danykafurt", 
"State": "NV" } },  { "Id": "E791196", "Name": "Denesik, Kessler and Rolfson",
...

Before we get around to creating this JSON text, we need to mention a couple of the differences between the XML and the JSON versions. For one, JSON doesn't have an equivalent to XML attributes, so the "Id" attribute was simply changed to a field named "Id". Second the repeating "Customer" element has been replaced by an array ( "Customer": [ ) and the individual customer elements simply become objects within that array. Each of those objects contains the fields "Id", "Name", and "Address" with "Address" being an object, which in turn contains the fields "City" and "State".

Creating JSON with YAJL

We could obviously create such a document "by hand" (i.e. by simply concatenating strings together) or by using a templating technique such as we have previously described for building XML. But since we also need to parse JSON as well as create it we decided to use Scott Klement's port of YAJL for both tasks. You can download the latest version of YAJL free of charge from Scott's site, and the install process is very simple and well documented. We should also mention that YAJL is very fast, and Scott has done his usual stellar job of "RPGizing" the API calls to make it easy to use. For example, his RPG interfaces automatically take care of translating between EBCDIC and UTF-8.

By the way, in case you were wondering, YAJL stands for "Yet Another Javascript Library"—not the most useful of acronyms! This package offers an API driven approach for JSON generation that is highly maintainable and easy to understand as you will see in a moment.

Let's take a look at the source code that we used to generate the Customers JSON document we showed above. We’ll look at the example in small pieces here but you can download the full source code here.

In the code below, we start at (E) with the control options (i.e., H specs for those who have not yet embraced fully-free RPG). We first specify that we want to use the YAJL binding directory (which is supplied in the download). The only other option that needs mention is the specification of DecEdit('0.'). This tells RPG to always include a leading zero in any %Char numeric conversions. This is required because a decimal number such as .75 is not valid in JSON. Numbers must start with a digit and therefore this must be expressed as 0.75. By specifying this control option we ensure that our program does it the right way.

At (F) we copy in all of the prototypes and constants for the YAJL routines. This is followed at (G) by the definition of the required error message field, which you will see in use later.

(E)    ctl-opt  DftActgrp(*No)    BndDir('YAJL')
                Option(*SrcStmt)  DecEdit('0.');
(F)   /copy yajl/qrpglesrc,yajl_h

       dcl-s  i             Int(5);
       dcl-s  wait          Char(1);
(G)    dcl-s  errMsg        VarChar(500) Inz;

Note that in this code we have indented the yajl_ calls to match the "shape" of the document.

The main processing begins at (H) with a call to the yajl_genOpen() procedure. This begins the JSON generation process. By passing the value *On we are telling YAJL to produce "pretty" formatted output. We would normally do this while in the development phase to make it easier to diagnose problems in the generation. Once the program is working and ready to roll into production we would simple change this parameter to *Off and JYAJL would then produce a compact non-formatted stream.

Next (I) we call yajl_beginObj() - passing no parameter signals the API to start a new JSON object that will contain the entire document - which I will refer to as the root object. This is immediately followed by a another call to the same API to begin the object named "Customers" (J). Since the customers object consists of an array of customer details the next thing to do (K) is to start the array by calling yajl_beginArray.

// Main Process Begins ...
      
(H) yajl_genOpen(*On);

(I) yajl_beginObj();  // Start the root object

(J)   yajl_beginObj('Customers');

(K)     yajl_beginArray('Customer');

At this point the base of the JSON document is in place. Now we just need to add the data for the individual customers. Since this is just an example program we want you to be able to copy it and "play" with it on your own system without having to worry about creating data files, etc. So we are simply using a For loop to generate the data for three customers, and in all cases the actual data values we are going to use are simply a character (%Char) version of the current For loop index. Obviously in a real working version the data would likely be supplied by an SQL query or RPG I/O operations.

Each element in the customer array is an object, so we start that at (L), followed by adding the values for the Id and Name fields (M). The address is an object containing the city and state so we begin that at (N) followed by adding the required fields (O). We then make two consecutive calls (P) to yajl_endObj() to terminate the address and array element objects before looping back to process the next customer.

       for i = 1 to 3;
(L)        yajl_beginObj();
(M)          yajl_addChar('Id': %Char(i));
             yajl_addChar('Name': %Char(i));
(N)          yajl_beginObj('Address');
(O)            yajl_addChar('City': %Char(i));
               yajl_addChar('State': %Char(i));
(P)          yajl_endObj();
           yajl_endObj();
       EndFor;

Once all customers have been output (i.e. the For loop ends) we then end the customer array (Q), followed by wrapping up the customer and root objects. The JSON document is now complete and ready to do whatever we wish with it. In the case of this simple example, and to make it easy to see the results of running the program, we have simply saved the whole buffer to an IFS file (R). Notice that in addition to providing the file name, we also supplied an error message field (errMsg). In the event of a failure while trying to write to the IFS, this would contain an appropriate error message; otherwise it will be an empty string. This possibility is tested at (S).

(Q)      yajl_endArray();

      yajl_endObj();   // End Customer
    yajl_endObj();   // End root

(R) yajl_saveBuf('/Partner400/JSONEX2.json': errMsg );

(S) if errMsg <> '';
      Dsply 'Ooppppssss - problem!' ' ' Wait;
    EndIf;

    *InLr = *On;

And that is all there is to it. A very simple process and one that remains relatively simple even when generating complex documents.

Additional Thoughts

The majority of the time when we generate JSON it will be in order to format a response to a browser's Ajax request or to build a request or response document for a web service. The only real difference between that and what we have done here would be to replace the call to yajl_saveBuf() with a call to yajl_copyBuf() which will take the JSON string and copy it into a regular RPG field ready to be passed to the appropriate output routine.

When developing JSON documents it can be very useful to take advantage of the IFS facility we have demonstrated here as this enables you to run the results through a JSON validator. That way you can be sure your document is valid before you start sending it out and getting into arguments with other developers as to whether the generation of the JSON or its subsequent processing is the cause of problems. There are many validators out there including JSONLint and FreeFormatter.com. The latter has the added advantage of allowing you to upload a file for checking.

In the next article we will look at how to process JSON with YAJL. In the meantime, if you have any questions or comments please let us know via the comments section below.

Jon Paris and Susan Gantner

An RPGer’s First Steps With JSON