I recently worked on an Interactive Brokers TWS API implementation in C++, and I wanted to share some thoughts on it while they are fresh in my mind.

Intro

The purpose of this post is to prove that IB’s TWS API has a socket implementation that is replaceable.  I made an implementation that retains all of the original encoders and decoders but uses a completely different socket and threading implementation.

In this post, I will show a completely stripped-down TWS API client to which one can add a socket implementation to.  I call it a “template” because the user is expected to add features to it, maybe rename it and the classes.

Then I talk a little bit about how the messages are encoded and decoded but not so much about the socket implementation itself.

Overview of the TWS API

The TWS API itself consists of a multi-language library, so one can import / include it directly in C#, Java, Visual Basic, C++, and Python.   It has a complete duplicate implementation in every language, including sockets and multithreading.  TWS itself was written in Java, so it is likely that the API was written in Java first, and the same exact Object-Oriented Programming model was applied to all of the language implementations to make them all familiar when hopping between languages. This likely means that if their engineers add a new feature to the API, that feature would be available in all languages.

Problem Summary

Unfortunately, I have personally encountered its socket and multithreading bugs.  At least I can confirm that the supplied implementation was inappropriate for my application.

So how does one handle this situation? There are different options, and I will list three of them:

1. Keep a copy of the TWS library and modify it until the bugs go away.  This puts a bandage on a much larger problem.
2. Write your own TWS library with an encoder, decoder, and sockets implementation.  Unless your application requires tight controls on memory allocation, this method has very poor ROI.
3. Keep the TWS classes, encoder, and decoder, but strip away just enough that the sockets implementation can be replaced.  This way, one can just plug a different sockets library in.

Option #3 was my weapon of choice. I ended up using Boost Asio, but I won’t discuss the implementation details here.

The Template

This header file sums up the complete structure of what the user is required to implement, member function for member function:

#pragma once
#include "EMessage.h"
#include "EClient.h"
#include "ETransport.h"
#include "EWrapper.h"
#include "EClientMsgSink.h"
#include "EDecoder.h"

class TwsTemplate;
class EClientSocket2;

// **********************
// Modeled after ESocket class.
class ESocket2 : public ETransport {

public:
	//overrides of ETransport virtual functions
	int send(EMessage* pMsg);
};

// **********************
// Modeled after EReader class.
class EReader2 {
public:
	EReader2(EClientSocket2* pC_);
	EClientSocket2* pC;
	EDecoder d;
};


// **********************
// Modeled after EClientSocket class.
class EClientSocket2 : public EClient, public EClientMsgSink {
public:
	EClientSocket2(EWrapper* pW);
	EWrapper* pW;

	//overrides of EClient virtual functions
public:
	void eDisconnect(bool resetState);
private:
	int receive(char* buf, size_t sz);
	void prepareBufferImpl(std::ostream&) const;
	void prepareBuffer(std::ostream&) const;
	bool closeAndSend(std::string msg, unsigned offset = 0);
	bool isSocketOK() const;

public:
	// overrides of EClientMsgSink
	void serverVersion(int version, const char* time);
	void redirect(const char* host, int port);

};

// **********************
// Modeled after TestCppClient class.
class TwsTemplate : public EWrapper {
public:
	TwsTemplate();
	~TwsTemplate();

	EClientSocket2* const pC;
	std::unique_ptr<EReader2> pR;

	/* recommended implementations:
	bool connect(const char * host, int port, int clientId = 0);
	void disconnect() const;
	bool isConnected() const;
	*/

	//overrides of EWrapper virtual functions
	//The prototypes file changes a lot, just include it.
	//This way, the *.cpp file will be easy to fix when you update the TWS API.
#define virtual
#define EWRAPPER_VIRTUAL_IMPL
#include "EWrapper_prototypes.h"
#undef EWRAPPER_VIRTUAL_IMPL
#undef virtual
};

And then there’s the *.cpp file, which I will link here.

As I said, this is a template.  If you compile it and try to run it, it will do absolutely nothing.  And that is the point: Go ahead and add as many member functions to it as you like.

I’ll note that the direct include of EWrapper_prototypes.h is to require implementations of every single function.  The reason is simple: Interactive Brokers has historically changed the EWrapper function interfaces.  If one updates the TWS API and gets compile errors, it should then be pretty easy to identify and fix the broken EWrapper implementations.

Socket Operations

For anyone who wants to implement the socket operations, it would be instructive to follow how the original TWS client works.  Here is a basic summary of the low-level stuff:

First, the client connects to the open TCP port.  The client must send four character bytes: {‘A’,’P’,’I’,0}.  (The fourth character is a null terminator.), and then “v<minversion>..<maxversion>”, such as “v100..180”. This is already implemented in EClient::sendConnectRequest(), and the EClient class is inherited by the above EClientSocket2 class, so it can be used.

After this, all messages after this are in the format of:
“<int_sizeofmessage><message_body_text>”.  This is for both incoming and outgoing messages.  The message body is in plain text, where each and every field will be parsed in sequence.  Empty fields will simply be null-terminated empty strings.  Sample code for encoding the header is in EClientSocket::encodeMsgLen().  For decoding the header, see EReader::readSingleMsg().

At this point, the socket should be reading for incoming messages.  In sequence:

1. Read four bytes from the socket to get the header.
2. Calculate bytes needed.
3. Read the number of bytes needed for the text message.
4. Parse the text message (using the EDecoder).
5. The resulting messages will call functions in the EWrapper function implementations.
6. Return to step 1.

Anyways, the server replies to indicate the server version. The EDecoder object will detect the server version using EDecoder::processConnectAck(), so now we know which version of the messages to send to the server.  (Aside: Now do you see why I kept the original encoders and decoders?)

At this point, we can call EClient::startApi(), which sends the “start api” message which tells the server what the client ID is.

Once all that is done, it is business as usual.  Send messages by calling the EClient:: functions, and receive messages with the EWrapper:: implementations.