OpenDAX

I've gotten to a pretty major milestone at this point. The system has been converted to the socket interface. The tag database has been converted to a new format and the API has been changed to reflect these modifications.

I've also added some features to a couple of modules. The modbus module reads and writes to tags that can be configured on a per command basis. The daxlua module now has multiple periodic tasks that can be configured and I also added the ability to pre-define some tags that will be exposed to the Lua script as global variables.

I'm thinking about changing the way that the Modbus Module handles data. Right now it simply has a large (size is configurable) data table. Each configured modbus command writes the results of it's query into some index of that data table. The data table is an array of DAX_INT's in the server. The bitwise modbus functions would write to bits in the data table that are indexed as bits. In other words bit 16 is the first bit in the second word of the data table.

The new database format as well as the new socket module interface made me rethink the data handling API in libdax. I have rewritten the three "raw" reading and writing functions (I've also renamed them). They are dax_read(), dax_write() and dax_mask(). They are the functions that actually get called to read/write raw data to/from the server. They make no assumptions about the underlying datatype. It is assumed that the module will send the data to these functions in the same format that they exist on the server.  The dax_mask() function is a writing function that also includes a bit mask.  The server will only change the bits in the database that are set in the mask.

Ever since I started dealing with these architectural problems concerning byte ordering and floating point formats, I have been struggling with issues in my present database format. Right now the real time data area is a large bitmap that contains the dynamic data. There is also an array of tag definitions that describe the data. The tag array describes the name, type, size of the data point (or tag) and also it's relative location in the real time database.

Okay, I've pretty well settled on the socket interface for IPC between the modules and the server. This will simply make the network distributed modules easier to build. I've decided to handle the endian issues by letting the server handle the data in any way that the server will handle it, then make the modules conform to that storage type.

During the registration process the server sends some different integers and floating point numbers to the module.  The module then determines whether or not its own storage format is different from the server and then determines what method it will have to employ to properly convert the data.

Okay so I grossly underestimated the amount of effort that it would take to change the system over to a sockets type IPC to replace the message queue. Actually replacing the message queue with a unix domain socket is not terribly difficult once the message buffering is done, but the the thought that I could use that same code for a remote module was naive.

Experienced network programmers may start laughing now. It's okay, this is my first time to try sending binary data over a network, and I figured it out BEFORE I wrote any code. For you Windows programmers out there let me explain the problem. Basically it's two things, endianess and structure packing.

I've forked the OpenDAX code and started working on version 0.2. There was never a release for version 0.1 but I'm going to be making a lot of changes to the code, so I changed the version. Primarily I've decided to change three things and each is pretty major and effects every part of the OpenDAX system, the server, the library and all the modules.

First, I'm going to change the IPC interface to sockets. I've been using a message queue for module to server interaction up until this point but it keeps causing me problems. I was hesitant to do this because it's going to be much more complex but in the end I think that it'll be more flexible and it'll get the code to where it needs to be quicker. Better to change it now while there are only a few modules.

I'm back to questioning the method that I am using for communicating between the OpenDAX server and the modules. I have been stuck on using a message queue for this communication. I have once again read a book (this one is rather authoritative) that said message queues should not be used for new projects. Uggghhh! I guess I probably knew that and have questioned the decision since the very beginning of the project. Given that I continue to question my choice of using message queues I've decided to write down what I think the pros and cons are of the queue and solicit comments on what I should be doing.

Pros

I had a few ideas last night for features that I think would be nice to add to OpenDAX. I need to write them down so I don't forget. As always comments, additions, complaints etc are welcome.