Introduction
This document is designed to explain the PathFinder product in more detail than is possible in the basic publicity leaflet, and to try and give more information about what the product can do and how it does it. It is aimed particularly at those familiar with data exchange requirements, and therefore contains a higher level of technical detail than the basic publicity.
What PathFinder can do
Describing PathFinder is complicated by the tendency of the IT world to repeatedly come up with new buzzwords for essentially the same techniques. We describe PathFinder as a data exchange tool—it can also be referred to as an EDI product or a middleware system.
Basically, PathFinder takes a file from one place, modifies it (if required) according to pre-configured rules, and delivers it to its destination, monitoring progress throughout the process. There are no restrictions on the type of files that can be handled, nor their quantity. Files can be transferred from one system to another on the same network, or from one remote partner to another.
In order to show how PathFinder works, this process can be split up into looking at the communications that PathFinder can use to send or receive files, the transformations that PathFinder can apply to the file, and the ability to both monitor the progress of files and to configure how they should be processed. These three areas are now covered in separate sections of this document. To conclude, some examples of how PathFinder is being used are given.
Receiving and sending files
There are a large number of ways that PathFinder can exchange data with the systems around it. These break down into three categories:
1. Listeners
PathFinder supports incoming calls by FTP, OFTP, SMTP and HTTP, providing specific server functionality for each protocol without the requirement for external software.
PathFinder’s FTP listener supports both standard FTP and FTPS (FTP over SSL). Files can be delivered or collected from the server. Although the remote can use any standard FTP or FTPS tool, and will think that they are accessing a normal FTP server, the advantage of the PathFinder FTP listener is that the remote never accesses the filesystem of the local machine. Instead, PathFinder simulates a virtual FTP server environment. This means that files sent by the remote are queued directly to PathFinder, rather than simply being written to a disk area. Since PathFinder is handling the entire transmission, and knows when transmission is complete, this guards against the standard FTP problem of attempting to access the file before it has been fully sent, and queues the file for processing with no delays whatsoever. When the remote tries to list the “directory” they are presented with, any files queued from PathFinder and waiting for the remote are shown to them as if they were a file listing, and can be picked up with the appropriate retrieve commands as if they were indeed files in a directory. However, because access to the actual file system is not permitted, this is a far more secure arrangement than the standard use of the operating system FTP server, which requires actual OS users to be set up.
PathFinder’s OFTP listener supports calls over both ISDN using the CAPI 2.0 protocol and TCP/IP. As would be expected, it supports the standard four data formats (unformatted, variable, text, fixed) and fully supports sending and receiving of files.
PathFinder can also handle files sent as email attachments, using either an SMTP listener or a POP3 collector (of which more later). Emails sent to the SMTP listener are scanned for attachments and the attachments loaded to PathFinder for processing.HTTP file transfer
Finally, PathFinder supports HTTP file transfer, including the AS/2 and RosettaNet protocols layered on top of the HTTP layer. It can handle both transfer by HTTP and HTTPS, and again requires no external web server—the HTTP protocol listener is responsible for the entire transaction. This makes monitoring and debugging communications sessions far easier, as there is only one place that needs to be checked—there is no possibility of files falling “down the hole” between a separate web server and an AS/2 communications system. The HTTP listener is also capable of receiving form output directly from a web form on another website, meaning that PathFinder can be used to process such information into the appropriate Interface file and load it to an application or database.
2. Collectors
Listeners are used to send files to PathFinder, but PathFinder can also use collectors driven by an internal scheduler to pick up files from remote or local network locations. PathFinder supports connections by FTP (including FTPS), POP3 and OFTP, as well as collections from local or network files and directly from databases. Collectors can be configured to pick up as frequently or infrequently as required, and also to notify the administrator by email in the event that a collection fails for a configurable number of retries.
Again, files picked up are automatically queued to PathFinder for immediate processing. The whole process, from the progress of a file download or upload to the exact protocol used in logging onto a remote site, can be viewed and monitored from the PathFinder client, since once again no third party software is used for any part of the communications—everything is done by PathFinder itself.
3. Application integration
In order to allow close integration to a range of applications that import and export data, files can also be queued to PathFinder from the command line. PathFinder’s open-Syntax translator (of which more later) can understand or produce standard export and import files for a wide variety of applications, removing the need for bespoke interfacing code.FTP connections Increasingly, PathFinder is being used by our customers to Interface one internal application to another, alongside its original use of providing an EDI Interface to external trading partners.
As can be seen, PathFinder has built-in communications to allow file transfer to and from as many sources as possible, by whatever means are most convenient to the systems and applications already. More information on how these communications are shown to the user can be seen in the later section.
PFTrans—the PathFinder translator
The engine of PathFinder is the PFTrans translator. This is the component that allows transformations of the data passing through PathFinder into the formats required by the file destination.
PFTrans is an open Syntax translator. This means that, unlike many of its competitors, it is not restricted by having to understand a particular message type. As the world moves from a rigidly standardised EDI structure to far more open data exchanges, it is important that PathFinder can support any document format—whether or not the programmers had heard about it at the time of the release!
PFTrans maps to the Syntax, not the message type. This means it can read from or produce any XML file, whether bespoke or defined by a product or industry sector, or anyone else. It can handle any delimited file, including the traditional EDI standards, CSV files, or anything else with defined field delimiters. And it can deal with any Flat file format, bespoke or otherwise. And any of these can be transformed to any of the others.
Transformation maps in PathFinder are produced by the PathFinder mapper tool included in the product. This allows such maps to be produced in a GUI environment. At its simplest, fields can be mapped from source to destination by simple point and click. However, if required for the transformation needed, far more complex logic can be applied, including conditional mapping, lookup lists, variables and other such techniques.
The challenge of a mapping tool is to allow simple maps to be produced without requiring programming or scripting knowledge on behalf of the user, while containing the high-level programming functionality to produce more complex transformations where needed. At this level, PathFinder’s mapper provides the ability to comment mappings, and also to disable and re-enable particular sections of the map during debugging. PathFinder maps also have built-in version control, and an integrated change history and documentation section. Those with a programming background will recognise the importance of these areas for producing maps that can be used long term and, should changes be required long after the map has been initially produced, will allow the PathFinder user to understand what different logical sections of the original map were written for.
To speed up map development, particularly for complex maps, the mapper includes its own test facility—there is no requirement for moving a partially developed map to the main server in order to check whether a particular mapping does the job it was intended to. The test screen allows the source data to be transformed using the partially-developed map in exactly the same manner as it would be on the main PathFinder server. In addition, the source and destination files can be displayed with field highlighting to show exactly what the map is doing, and the Translation stages are broken down into individual mappings so that the process can be followed through stage by stage.
PathFinder’s mapper and PFTrans translator between them allow the user to control exactly what transformations they require. They assist the user as much as possible in map development, to try and speed up what can be a complex task. And, once again, they have no restrictions on the documents that can processed—if it can be defined in a standard Syntax, PathFinder can work with it.
Monitoring and configuring PathFinder
The cleverest data exchange system in the world is no use unless the user can view easily what is happening in it. For all but the simplest systems, users need to know whether an expected message has arrived yet, or when a despatch advice message was sent out, or whether the order for a given part number has been loaded into the system. And, in the event of problems, the user needs to be able to quickly identify and view a failure, understand the reason for it, be helped to resolve it, and then to be able to reprocess the file successfully. The PathFinder client has been explicitly designed so that all these operations, plus many more, can be conducted from the client rather than needing any operation on the server itself.
There are two specific screens in the client that are used to monitor the system. Using the data archive screen, the user can see what files have passed through and are currently being processed by the system, and exactly what happened to them at every stage. Every file processed by PathFinder is available from the archive, and can be filtered by date, direction, status, and originator, as can be seen in the above screenshot. Once the file that needs to be looked at is selected, the user can display the file as it arrived at PathFinder and as it left (before and after any transformations). In addition, log information for every stage of the file’s progress can be displayed, at the chosen log level to avoid the user being swamped with information. And finally, a view of all communications routes that have been used to send the file on to its various destinations can be shown. Menu options on this screen allow reprocessing, resending and cancellation of any part of the process, and the results of this can be immediately seen.
In addition, the data archive has a built in search facility. It is possible to search for messages containing a given part number, or a specific order number, or any other data field, along with the other possible filters. This can be used, for example, to display all inbound order messages from a given company in the last month that contain a specific part number—there are many other uses that this can be put to.
The other screen that is principally used to see what the system is doing is the communications tracking screen. Again, due to the fact that PathFinder handles all communications itself without using external products, all communications sessions can be viewed through the same Interface with the same style of logging.
Every communications session, whatever the protocol, destination or direction, is logged and can be easily displayed. There is no requirement to look through huge log files—not only can log information be displayed per file, per process or per communications session, but again it is also logged at four different levels of importance—so the user can show only what information they need to know when browsing the system, but drill down into the detail easily in the event of a problem or question.
For each communications session, the user can display either the log information associated with the session, or the files sent and received during it—and clicking on the files takes the user straight to showing that file in the data archive, to allow continuity when tracing a file. Problem sessions can be displayed and, since PathFinder’s lowest logging level shows the actual protocol layer, the problems can be quickly diagnosed.
Examples of PathFinder usage
These are all examples of existing customer usage of PathFinder:
General data exchange:
Interfacing of web-produced data between a third party web store and the main goods supplier.
Receipt of invoices in PDF format from a supplier sent as email attachments, and routing into a directory for internal manual processing.
Interface of two existing in-house applications for bi-directional exchange of data using existing export and import formats.
Electronic Data Interchange (EDI):
Replacement of traditional EDI systems handling EDIFACT, Tradacoms, ANSI X12 and other local and worldwide formats. As well as handling the existing requirements, sites using PathFinder for EDI know that future expansion into other wider forms of messaging and communications will not require a change of product.
Access to central marketplace using RosettaNet protocol, to exchange business documents with other suppliers and customers.
Conclusion
Most documents of this type finish by telling you that the product in question will “revolutionise your business”. We make no such pointless claims for PathFinder. However, what it will do is to look after all your internal and external data exchanges in one package, reducing the time required to setup, monitor, and track your various documents, and increasing the reliability. The time you will save as a result will certainly help your business.
We hope that this document has allowed you to discover more about what PathFinder may be able to do for your business. If you have any further questions, please contact Orion Consulting using the link on the right or by telephone to +44 (0) 1572 771316.