Inter-process Messaging Application Platform
Distributed Application System Development
Contact: Sally Lee
In today's Internet explosive information age, business application systems are deployed in a distributed multiple hosts environment. Application processes run on different or same hosts need to exchange messages with each other. Most inter-processes communication architectures and products exist today, e.g. HTTP, JAVA RMI, CORBA and DCE, emphasize on Client-to-Server messaging services which are modeled as Remote Procedure Calls (RPC). In RPC messaging paradigm, a client process always initiate a "request" and wait for a single "response" from the server process. Although RPC model works well for most user front-end processes, it becomes awkward for the back-end server-to-server communications. Server-to-server communications implies that two processes establish connection to each other with two-way concurrent message exchanges in a peer-to-peer fashion. The traffic volume in a server-to-server communication environment usually is much higher than the traffic volume of client-server messaging. To implement high volume concurrent message exchanges in the RPC programming paradigm, the application process would have no choice but become multi-threaded. Multi-threaded application programs usually involve data locking and data contention. Data locking among threads could cause dead lock. Data contention among threads could cause data corruption. Creating test cases to simulate concurrent thread executions with different speeds is complicated. It is much more difficult to prove the correctness of a multi-threaded application than traditional single threaded application.
The Inter-process Messaging Application Platform (IPMAP) offered by SynerCom Inc. is a library of Application Program Interfaces (APIs) which provides for developers of distributed application systems with an easy programming environment for reliable high volume server-to-sever message exchanging services. The IPMAP software platform provides JAVA and C/C++ application programs with APIs to perform high volume concurrent data exchanges among a large number of processes. It does not demand multi-threaded application programming to reach high performance throughputs. Using IPMAP platform, processes coded in either JAVA or C/C++ may exchange messages in the same host or different hosts seamlessly. The platform has been used as the base infrastructure of several distributed application development projects successfully. SynerCom offers customized programming services to meet our customers’ specific needs for their businesses. IPMAP source code licensing is available.
A typical distributed application systems with mixed C/C++ and JAVA server processes and Web Server Servlets/CGI is depicted in the following figure.
The following list describes the benefits and features of IPMAP:
Simple and low cost development & deployment
The platform software is available in source code or binary form format. To compile the C version of the source code, only an ANSI C compiler is needed. It does not require any third party software packages for compile and link. The C version could be ported to other vendor operating system with only C compiler and TCP socket supports. JAVA version can be run on any vendor platform, which implement the JAVA Virtual Machine. The software has been fully tested on both Sun SPARC and Intel platforms running under Sun Solaris operating systems.
The platform software uses TCP/IP as the underlined session and transport protocol. TCP/IP is the industrial de-facto standard protocol for Internet and corporate enterprise networks. Most of vendor host operating systems and networks support TCP/IP protocol. Deploying IPMAP based application requires no third party software and licensing.
Easy Programming Environment
C/C++ programming with single threaded C APIs
Unlike most RPC based programming packages such as JAVA RMI, CORBA and DCE, the platform APIs do not require application programs to use multiple-threaded programming techniques to support multiple connections with concurrent events. Multiple-threaded application programming demands sophisticated concurrency control from application programmers. Multiple-threaded application programs are difficult to test and debug. Many UNIX system and function calls are not thread-safe. IPMAP APIs allow programmers to send and receive messages through multiple TCP connections in a single threaded environment. The IPMAP library routines perform TCP session connect/accept, message send/receive in a 100% non-blocking event wait model in single thread seamlessly.
JAVA programming with transparent multiple threads and synchronization support
JAVA provides native threads by the language syntax. All JAVA system objects are thread safe. In order to achieve maximum throughputs, the JAVA version of the IPMAP uses JAVA threads with JAVA synchronized object/method support. However, with IPMAP JAVA objects (APIs), the complexity of the multiple-threaded programming is transparent to the using JAVA application programs. The JAVA version of the IPMAP library objects performs TCP session connect/accept, message send/receive in a 100% non-blocking event wait model.
Automatic session management
Active TCP connections can be established by application with a single API call. Passive TCP connections will be handled automatically without application intervention. The complexity of TCP socket programming for connection establishment is completely shielded by IPMAP. Active TCP connections need only be established once. If a TCP session failed for any reason, this session will automatically be recovered by reconnects.
Non-blocking message send & receive
To avoid blocking delay, messages send to TCP connections will be buffered with no I/O delay. The actual message sent operations are performed automatically in the background.
Data received from multiple TCP sessions will be collected in the background in a non-blocking read fashion. When all the data segment for a complete message is collected, the whole message will be presented to the application through API. Application programs will be notified when there is a complete message arrived from any of the established TCP sessions.
Event wait with time-out API
Application C program will be notified with all registered events through a single event-wait API. The events include ready-for-read on any TCP session and user time-out. Application program does not need to "poll" for any events. No CPU time is required when there is no event to be processed.
The JAVA version of the APIs uses "JAVA Interface" to invoke callback methods provided by the using application objects as notification of all events.
When a process is receiving data faster then it could send due to downstream server process delay. It is important to apply "back pressure" through the TCP connection to upstream process for a slow down. APIs for application programs to instruct IPMAP to apply "back pressure" to upstream process for flow control are provided.
Memory and disk queue by applications
APIs for application programs to create and manage message queues are provided. The application may stored and retrieve messages in a FIFO (first in first out) fashion in memory queue. The application program can choose to store the current image of memory queue into disk. The disk queue may be used by application as an extended queue. The disk queue can also be used to store and recover the queue when application process restarts.
Message data trace
For development debugging and production site trouble shooting, it is important to have data trace in-between processes. The IPMAP can be configured statically or dynamically to provide trace data for messages exchanged in-between processes. The trace data is stored in trace files. The trace data files can be archived with date and time. A utility program is provided to generate formatted trace report from trace files.
Process and connection event logging
A generic message logging utility is included in the platform API for application programs to log any error or status messages to the log files of each process. The IPMAP uses API to record significant protocol events – e.g. connect, disconnect, session recovery, etc. The IPMAP maintains two log files, Current and Old, which are used by each process. When the Current log file reaches a pre-configured size, it will be renamed to Old. Then, a new Current file will be created. These log files are useful in development debugging and production site trouble shooting.
Connection status reporting by API and web browser
Application program may retrieve TCP connection status through APIs. The administrator may request all TCP connection status for each process via Web Browser using HTTP protocol – i.e. Netscape or Internet Explorer.
Light-weighted Process Directory Services and Run Control
A light-weighted Directory Server is provided for processes to register their locations. Processes may inquire the Directory Server to locate target destination processes. The Directory Server can also be used to track process run status and perform system shutdown if needed.
Simple XML Encoding and Decoding
There is XML encoding and decoding APIs for application to use to package structure of data for data exchanges.
Single process benchmark test on Intel Pentium-II 400Mz CPU platform is 500 message per second with constant message size 550 bytes each.