QuickFIX: Load a message from the logs

QuickFIX: Loading and Replaying Messages from Logs

Learn how to effectively load and replay FIX messages from QuickFIX log files for debugging, testing, and analysis purposes.

QuickFIX is a robust open-source engine for the Financial Information eXchange (FIX) protocol. During development, testing, or production support, it's often necessary to inspect past FIX messages. QuickFIX logs all incoming and outgoing messages, providing a valuable historical record. This article will guide you through the process of programmatically loading these messages from QuickFIX log files, enabling you to replay them, analyze specific scenarios, or integrate them into custom testing frameworks.

Understanding QuickFIX Log Formats

QuickFIX typically generates two types of log files for messages: FIX.4.2-SENDER.TARGET.messages.log (or similar, depending on FIX version and session IDs) for session-level messages and FIX.4.2-SENDER.TARGET.event.log for session events. The message logs contain the raw FIX messages, prefixed with a timestamp and an indicator for incoming (<) or outgoing (>) messages. Understanding this format is crucial for parsing the logs correctly.

20230101-10:00:00.123 : <8=FIX.4.2|9=65|35=A|49=TARGET|56=SENDER|34=1|52=20230101-10:00:00.000|98=0|108=30|10=191|
20230101-10:00:00.456 : >8=FIX.4.2|9=93|35=A|49=SENDER|56=TARGET|34=1|52=20230101-10:00:00.000|98=0|108=30|10=191|
20230101-10:00:01.789 : <8=FIX.4.2|9=123|35=D|49=TARGET|56=SENDER|34=2|52=20230101-10:00:01.000|11=ORDER123|21=1|38=100|40=1|54=1|55=IBM|59=0|60=20230101-10:00:01.000|10=000|

Example QuickFIX message log entries. Note the timestamp, direction indicator, and SOH-delimited FIX message.

Programmatically Loading Messages

To load messages programmatically, you'll need to read the log file line by line, parse each line to extract the raw FIX message string, and then use QuickFIX's quickfix.Message class to construct a message object. This allows you to access individual fields and perform validation.

Workflow for loading FIX messages from a log file.

Python Example

import quickfix

def load_messages_from_log(log_file_path): messages = [] with open(log_file_path, 'r') as f: for line in f: # Example log line format: 20230101-10:00:00.123 : <8=FIX.4.2|... parts = line.split(' : ', 1) if len(parts) > 1: direction_and_message = parts[1] # Remove direction indicator (< or >) and any leading/trailing whitespace raw_fix_string = direction_and_message[1:].strip() # Replace '|' with SOH character (ASCII 0x01) raw_fix_string = raw_fix_string.replace('|', '\x01') try: msg = quickfix.Message(raw_fix_string) messages.append(msg) except quickfix.InvalidMessage as e: print(f"Error parsing message: {e} in line: {line.strip()}") return messages

Example usage:

log_file = 'FIX.4.2-SENDER.TARGET.messages.log'

loaded_messages = load_messages_from_log(log_file)

for msg in loaded_messages:

print(f"Loaded message type: {msg.getHeader().getField(quickfix.MsgType()).getValue()}")

C++ Example

#include <quickfix/Message.h> #include <quickfix/Field.h> #include <quickfix/Exceptions.h> #include <quickfix/Utility.h> #include #include #include #include

std::vectorFIX::Message loadMessagesFromLog(const std::string& logFilePath) { std::vectorFIX::Message messages; std::ifstream logFile(logFilePath); std::string line;

if (!logFile.is_open())
{
    std::cerr << "Error: Could not open log file " << logFilePath << std::endl;
    return messages;
}

while (std::getline(logFile, line))
{
    // Example log line format: 20230101-10:00:00.123 : <8=FIX.4.2|...
    size_t colonPos = line.find(" : ");
    if (colonPos != std::string::npos && line.length() > colonPos + 4)
    {
        std::string rawFixString = line.substr(colonPos + 4); // Skip timestamp, ' : ', and direction char
        // Replace '|' with SOH character (ASCII 0x01)
        FIX::replace(rawFixString, "|", "\x01");
        try
        {
            FIX::Message msg(rawFixString);
            messages.push_back(msg);
        }
        catch (const FIX::InvalidMessage& e)
        {
            std::cerr << "Error parsing message: " << e.what() << " in line: " << line << std::endl;
        }
    }
}
return messages;

}

// Example usage: // int main() // { // std::string logFile = "FIX.4.2-SENDER.TARGET.messages.log"; // std::vectorFIX::Message loadedMessages = loadMessagesFromLog(logFile); // for (const auto& msg : loadedMessages) // { // FIX::MsgType msgType; // msg.getHeader().getField(msgType); // std::cout << "Loaded message type: " << msgType.getValue() << std::endl; // } // return 0; // }

Java Example

import quickfix.Message; import quickfix.FieldNotFound; import quickfix.InvalidMessage; import quickfix.field.MsgType;

import java.io.BufferedReader; import java.io.FileReader; import java.io.IOException; import java.util.ArrayList; import java.util.List;

public class FixLogLoader {

public static List<Message> loadMessagesFromLog(String logFilePath) {
    List<Message> messages = new ArrayList<>();
    try (BufferedReader reader = new BufferedReader(new FileReader(logFilePath))) {
        String line;
        while ((line = reader.readLine()) != null) {
            // Example log line format: 20230101-10:00:00.123 : <8=FIX.4.2|...
            int colonPos = line.indexOf(" : ");
            if (colonPos != -1 && line.length() > colonPos + 4) {
                String rawFixString = line.substring(colonPos + 4);
                // Replace '|' with SOH character (ASCII 0x01)
                rawFixString = rawFixString.replace('|', '\u0001');
                try {
                    Message msg = new Message(rawFixString);
                    messages.add(msg);
                } catch (InvalidMessage e) {
                    System.err.println("Error parsing message: " + e.getMessage() + " in line: " + line);
                }
            }
        }
    } catch (IOException e) {
        System.err.println("Error reading log file: " + e.getMessage());
    }
    return messages;
}

// Example usage:
// public static void main(String[] args) {
//     String logFile = "FIX.4.2-SENDER.TARGET.messages.log";
//     List<Message> loadedMessages = loadMessagesFromLog(logFile);
//     for (Message msg : loadedMessages) {
//         try {
//             System.out.println("Loaded message type: " + msg.getHeader().getString(MsgType.FIELD));
//         } catch (FieldNotFound e) {
//             System.err.println("MsgType field not found in message.");
//         }
//     }
// }

}

Replaying and Analyzing Messages

Once you have loaded the messages into quickfix.Message objects, you can perform various operations. You can iterate through them to find specific message types, extract field values, validate message structure, or even replay them against a QuickFIX acceptor or initiator for testing purposes. For replaying, you would typically send these messages through a QuickFIX session, ensuring sequence numbers and timestamps are handled appropriately.

1. Identify Log File

Locate the relevant QuickFIX message log file (e.g., FIX.4.2-SENDER.TARGET.messages.log) that contains the messages you wish to load.

2. Implement Log Parser

Write code (using Python, C++, Java, etc.) to read the log file line by line. For each line, extract the raw FIX message string by stripping the timestamp and direction indicator.

3. Convert SOH Delimiter

Replace the visible | character (or whatever your editor displays) with the actual SOH character (ASCII 0x01 or \x01 in code) before creating the quickfix.Message object.

4. Create QuickFIX Message Objects

Instantiate quickfix.Message objects using the parsed raw FIX strings. Handle InvalidMessage exceptions for malformed messages.

5. Process or Replay Messages

Iterate through the list of loaded quickfix.Message objects to analyze their content, extract specific fields, or integrate them into a testing framework for replaying.