Discover how simple it is to create a generic handler
This month, we’re focusing on our first real foray into the world of Open Access for RPG (OAR) handlers. In previous articles, we talked about OAR in general terms; this time we’ll get down to specifics.
You might recall that in our earlier articles “Get More from RPG” and “Making RPG Even More “Special”,” we described using RPG’s SPECIAL files and how OAR could alleviate some of their shortcomings. What we didn’t expect as we leapt headlong into our OAR explorations was how just simple it would be to create a generic handler and how much power you can get from relatively little effort.
To demonstrate this, rather than starting from the SPECIAL file program, we decided instead to create a new one that would have greater utility value. The program in question creates a Comma Separated Value (CSV) file containing one line for each record written by the RPG program. Yes, we know you can already achieve this by creating an output file and using Copy To Import File (CPYTOIMPF), but haven’t you ever wished you could just do it directly from within your RPG program? We have.
Both sample programs will use the same basic rules for the file’s output format. Character fields will be trimmed of trailing spaces and enclosed in double quotation marks. Numeric and date fields will be output as-is and any other data types (e.g., timestamps) will be ignored. Additionally, we’ll use a comma as the field separator following all fields except the last in the record.
A New SPECIAL File
You’ll recall that to use the Special File program we simply declare the file (an output file in this case) as RPG device “Special” and supply a program name via an F spec keyword. The F spec in our “calling” program looks like this:
FIFS_OUT1 O E Special PgmName('IFSWRITE')
Our special file is written to expect data in a specific format, so the program declares an externally described data structure as the template for the record buffer that’s passed as the fourth parameter (A). In addition, we /Copy’d the prototypes for the IFS APIs from the IBM supplied prototypes in QSYSINC (B). We’ve also defined a few work fields and constants (C).
So the relevant D specs of the program look like this, omitting, in the interest of space, the Prototypes for the program and its internal subprocedures. (In 7.1 we’d be able to omit these from the source member if we wanted—hurray!)
D IFSWrite PI
D Action 1A Const
D Status 1A
D Error 5S 0
(A) D RecData LikeDS(ExtDef)
(B) /copy qsysinc/qrpglesrc,ifs
(A) D ExtDef E DS EXTNAME(IFS_OUT1) Template
(D) D FileHandle S 10I 0
// Constants for SPECIAL file operation requests
(C) D OPEN_Request C 'O'
D WRITE_Request C 'W'
D CLOSE_Request C 'C'
The mainline logic flow for this program is very simple since all it needs to do is determine which operation is requested and invoke a procedure to handle each one. Since it’s almost identical in this respect to the previous examples, we won’t go into detail.
If Action = OPEN_Request;
FileHandle = openFile('/partner400/TestSG');
elseIf Action = WRITE_Request;
WriteFile(FileHandle);
elseIf Action = CLOSE_Request;
closeFile(FileHandle);
*InLR = *On;
EndIf;
Return;
All of the real work happens in the procedures, but even that logic is quite simple. We’ll show you the OPEN_Request procedure. The Open action will call our program in the case of either an implicit RPG or user-controlled open request. We won’t go into the meaning of the content of the flags and mode fields. We’ll also leave the details the parameters passed to the IFS open API for your own study. To learn more about the IFS APIs, we recommend Scott Klement’s free tutorial.
Discover how simple it is to create a generic handler
Actually, the fact that we’re not explaining the details of the IFS interfaces offers a great opportunity to remind you that one of the greatest benefits of using either Special Files or RPG OA is that it masks the complexity or just plain ugliness of functions—such as system APIs—from the programmers in the shop. They don’t need to know the details any more than they need to know how CHAIN works under the covers. Only one person needs to know how to write and maintain the program that does the IFS work; the rest simply treat IFS stream files as “just another file” and read and/or write to them normally.
If the open() API returns an error indication (a -1 value), the Special File program returns the error status value of 2 to the RPG runtime. If there’s no error, a very important piece of information is returned to the main part of the program—the file handle. This will be required to be passed on any future requests (such as write or close) to this IFS file.
D openFile PI 10I 0
D IFSFileName 30A Const
D Flags S 10U 0
D Mode S 10U 0
D OpenResult S 10I 0
/FREE
Flags = O_CREAT + O_WRONLY + O_CCSID + O_TRUNC
+ O_TEXTDATA + O_TEXT_CREAT ;
Mode = S_IRUSR + S_IWUSR + S_IRGRP + S_IROTH;
OpenResult = open(IFSFileName : Flags : Mode : 819 : 0);
If OpenResult < 0;
Status = '2';
Else;
Status = '0';
EndIf;
Return OpenResult;
/END-FREE
The procedure to write each record to the IFS simply needs to format the data and write it using the Write API. In our case, the program is hard-coded to expect the data buffer to be in the format of a specific externally described file. So we know each field’s name and data type and can deal with it appropriately, such as putting quotes around character fields, etc.
D writeFile PI
D FileHandle 10I 0
D WriteResult S 10I 0
D CSVData S 50A Varying
D CRLF c x'0D25'
D Comma C ','
D Quote c '"'
/FREE
(E) CSVData = Quote + RecData.CustNo + Quote + Comma
+ %Char(RecData.CRLimit) + Comma
+ %Char(RecData.LastOrder:*usa) + CRLF;
WriteResult = Write(FileHandle: %addr(CSVData:*Data): %Len(CSVData));
If WriteResult < 0;
Status = '2';
Else;
Status = '0';
EndIf;
/END-FREE
The handling of the close request simply involves calling the close() API, so we won’t take time studying it. While the program wasn’t hard to write, it must be customized for each and every record format you wish to use. As we noted in earlier articles, it’s not actually impossible to code a more generic solution, but it comes close—particularly when compared with the OAR solution, as we’ll soon see.
Discover how simple it is to create a generic handler
The OAR Approach
Our OAR example has identical functionality to the SPECIAL file version except that it also allows for the specification of the IFS file name rather than hard coding it as the SPECIAL example did. The basic logic flow is very similar—the handler will be passed a series of requests to open, write and close the file and will perform those operations on behalf of the caller.
OAR really shines when it comes to providing a generic solution because all of the field information you could ever need (and more besides) is available to you. This relieves you from having to pre-arrange with the caller your data format as you had to with the SPECIAL file. So, let’s start our examination of the OAR version with the code that works at the field level and then work backwards to see how we reached that point.
OAR’s Field-Level Information
The field-level information supplied is referred to in the documentation as the Names/Values structure, and IBM supplies the layout in the source member QRNOPENACC that’s shipped as part of the QOAR library. The definition of the individual field details is in the template DS QrnNameValue_T. There will be one instance of this DS for each field in the record format being processed. The definition you incorporate into your program, however, is QrnNamesValues_T, which in turn contains a reference to QrnNameValue_T.
The definition of QrnNamesValues_T looks like this:
D QrnNamesValues_T...
D DS QUALIFIED TEMPLATE ALIGN
D num 10I 0
D field LIKEDS(QrnNameValue_T)
D DIM(32767)
As you can see, it contains a count field (num) followed by an array of the QrnNameValue_T Data Structures (field). The count determines how many active field structures there are in the DS array. To incorporate this structure into our program, we simply coded the following:
// Field Names/Values structures
D nvInput ds likeds(QrnNamesValues_T)
D based(pNvInput)
The result is that we reference the count field as nvInput.num and the components of the field structure as nvInput.field(x).xxxx where xxxx is the relevant field name. If at this point some of you are thinking “what’s with all these periods in the data names?” this might be an opportune moment for you to do a quick review of the definition and use of qualified data structures. You can find some of our thoughts on the subject here.
Having defined the field information we can now use it. Our main field-processing loop looks like this:
// Process all fields in the record
(F) For i = 1 to nvInput.num;
(G) pvalue = nvInput.field(i).value; // set up to access data
(H) If ( nvInput.field(i).dataType = QrnDatatype_Alpha )
Or ( nvInput.field(i).dataType = QrnDatatype_AlphaVarying);
buffer += quote
+ %trimR( %subst( value: 1: nvInput.field(i).valueLenBytes ))
+ quote;
(I) ElseIf ( nvInput.field(i).dataType = QrnDatatype_Decimal );
buffer += %subst(value: 1: nvInput.field(i).valueLenBytes);
ElseIf ( nvInput.field(i).dataType = QrnDatatype_Date );
buffer += %subst(value: 1: nvInput.field(i).valueLenBytes);
EndIf;
If i <> nvInput.num; // Add comma after every field except the last
buffer += comma;
EndIf;
EndFor;
The most obvious difference between this and the version in the SPECIAL file program (E) is the additional logic required to handle fields generically. The previous version “knew” what formatting was required for a specific named field; this version determines the formatting based on the field’s type. It doesn’t have to know the field name, although as it happens it does.
The logic is driven by a For loop (F), which will step through each of the fields in the Names/Values structure array. The loop is controlled by the count field nvInput.num we already mentioned. The first thing we must do in the loop is to gain access to the field’s data by using the pointer in nvInput.field(i).value to set the pointer pvalue (G). The actual data can then be accessed via the field “value”:
D value s 32470a Based(pvalue)
We can use this single definition because the field’s value is always supplied in what IBM describes as “Human readable form,” which is to say in character rather than packed decimal or binary. In this regard, the result is the same as we get when we reference such fields in a printer or display file. Don’t worry too much if you don’t feel you fully understand how this “pointer stuff” works. It will always be the same; just clone something similar into every handler you write and all will be well.
The field’s type is contained in the variable dataType and compared against a series of IBM-supplied constants such as QrnDatatype_Alpha and QrnDatatype_AlphaVarying (H). Once the data type is matched, the appropriate formatting can be applied. The %Subst BIF is used to extract the actual field value, the length being supplied by the field valueLenBytes. This string is then added to the output buffer with the appropriate additional formatting.
So far, we’ve seen how the Name/Value information is used to control the output. But how did we get there?
Discover how simple it is to create a generic handler
Handling the I/O Requests
The first time the handler is called will be to process an open request. Just as with SPECIAL files, this happens whether the file is opened explicitly or under RPG’s control. The handler is passed a single parameter defined in the IBM /Copy members by the template QrnOpenAccess_T, a lengthy DS containing some 60 or so elements, some of which are nested data structures. But never fear—we need very few of them in this case, and you’d have to be writing a handler that could process everything from a keyed disk file to a display subfile before you ever came close to using all of them in a single handler.
First, we must gain access to any state information that we’ve stored. That’s the purpose of the code at (J) and we won’t comment more on this until we return to the topic when we describe the actual process of opening the file.
At (K) we begin the process of determining what type of request we’re handling. We coded the test for the WRITE operation first since that’ll be the most common request. We’ve already looked in detail at what happens during the write process, so the only thing to note is that the code at (L) is responsible for giving us access to the Name/Value data by setting the basing pointer (pnvInput) for the structure (nvInput) prior to calling the write routine.
Opening the File and Creating the State Information
The test and processing of the open request occurs at (M). First, we set the useNamesValues indicator to request that we receive the Names /Values information with all subsequent requests. Once set during the open process, this shouldn’t be changed.
Next, we allocate some storage to retain state information for the file and inform RPG where in memory this information is stored. In essence, this is a very simple process but can be confusing if you’ve never had to worry about such things. The simple way to think of it is this: Our handler may be required during the course of a job (indeed during the run of a single program) to process multiple files. Perhaps the same fundamental data is to be output in different formats to two different CSV files. Somehow the handler needs to be able to determine which physical file to use for any given file request. As it happens, when we use the IFS open() API, we receive a “handle” to the file we have opened—that’s the information we need to remember and associate with the current file. Since we have the file name, etc., supplied in the info block, we could of course use that and store it in an array, etc. But because this is a requirement for all handlers, RPG OA supplies a simple, elegant way of handling this.
RPG guarantees that if we place a pointer to our file-specific storage in the stateInfo field of the info structure then RPG will return that same pointer to us on every subsequent I/O request. Since all we need to store is the handle to the IFS file, we simply dynamically allocate a piece of storage large enough to contain it (N) and then store the pointer in the stateInfo field. When we subsequently open the file (P), the file handle (fileHandle) is stored in this allocated memory. On all subsequent I/O operations, we simply use the pointer that RPG returns to us to reset our access to this file handle (J). Again, don’t worry if you find this a little confusing. It’s something that every well-written handler must do, but (apart from the details of what data needs to be stored) the process is always the same.
(J) pfileHandle = info.stateInfo;
(K) If info.rpgOperation = QrnOperation_WRITE;
// Set up access to Name/Value information
(L) pnvInput = info.namesValues;
// Write error is unlikely but signal it if it occurs
If ( writeFile(fileHandle) = fileError );
info.rpgStatus = errIO;
EndIf;
(M) elseIf info.rpgOperation = QrnOperation_OPEN;
// Specify that we want to use Name/Value information
info.useNamesValues = *On;
// Allocate the storage for the file handle and store the pointer
// in the info area. That way RPG can associate the pointer with
// the specific file and give it back to us on each operation.
(N) pfileHandle = %Alloc(%Size(fileHandle));
info.stateInfo = pfileHandle;
// Ensure that file handle is zero before attempting open()
clear fileHandle;
(O) pIfs_info = info.userArea;
(P) fileHandle = openFile (ifs_info.path); // Open file
if fileHandle = fileNotOpen;
info.rpgStatus = errImpOpenClose; // Open failed
EndIf;
(Q) elseif info.rpgOperation = QrnOperation_CLOSE;
closeFile (fileHandle);
// free the state information and null out the info pointer
dealloc(n) pfileHandle;
info.stateInfo = *null;
else;
// Any other operation is unsupported so notify RPG
(R) info.rpgStatus = 1299; // general error status
endif;
Discover how simple it is to create a generic handler
Closing the File and Tidying Up
At (Q), we find the logic for closing the file. Because we dynamically allocated the storage for the file handle (i.e., the state information), we must release it now that we’re closing the file. This is the purpose of the dealloc() operation. Just to be on the safe side, we also set the stateInfo pointer to null so that there’s no chance of ever using storage that no longer “belongs” to us.
The final part of the code (R) sets an error status should any other type of I/O operation be attempted, such as a READ. We need to consider this possibility because such attempts can only be caught at run time. If this handler were to be specified for use with an input only file, for example, that error couldn’t be detected at compile time, only at run time. That’s really all there is to it.
One Last Thing
In our enthusiasm for describing the mechanics of the handler we forgot one small detail—how to specify that the handler is to be used for a specific file. As you’ll see, there’s so little to do that it’s no wonder we forgot!
To use the handler, we simply specify the keyword on the F-spec and identify the program or subprocedure we wish to use as the handler. In our example, we’ve used a program as the handler and also specified the additional parameter that will be used to pass the IFS file name to the handler. The rest of the program almost identical to the one that used the SPECIAL file. The F-spec looks like this:
FIFS_OUT1 o e Disk Handler('HND_IFS_J2' : ifs_info1)
F UsrOpn
Note that specifying UsrOpn for the file is not compulsory, but without it we’d have to have passed in the IFS file name as a parameter as there’s no other way to set the value into the ifs_info DS before the open operation.
Discover for Yourself
As you’ll discover if you download the code and try it, by the simple expedient of changing the file name used in the F-spec for the IFS output definition, we completely change the output of the handler. The new file layout will govern the IFS file output. You might also want to try using the handler for two different files in the same program; you’ll find it works beautifully with no additional coding required in the handler to accommodate it.
We hope this example will encourage others to start working on their own handlers. They deal with all of the issues associated with SPECIAL files while retaining the essential capability to mask the complexity of the underlying functions with the familiarity of RPG’s regular op-codes. This is the true power of the OAR system.
Oh! One last thing. We hope to introduce some more handlers in the future as well as revisiting this one to improve its error handling and reporting capabilities.