Moving the Dovetail Twitter Integration To Jabber/XMPP
Quickly after creating a Twitter Integration with Dovetail I ran into rate limiting issues using Twitter’s API. Twitter currently only allows 70 requests per hour. Here is their guidance…
If you are developing an application that requires more frequent requests to the Twitter API, please contact us and we’ll see what we can do. We maintain a white list of known-good screen names who have high-volume API needs. We also suggest researching Jabber/XMPP if your application needs to deliver a near-realtime experience (for example, a bot that replies to direct messages).
I emailed Twitter support asking nicely for higher volume API usage but still haven’t gotten a response. Then I realized that if we want to release this Dovetail/Twitter integration to our customers we it need to work within the rate limiting guidelines, so I listened to Twitter’s guidance and researched using Jabber/XMPP.
The first thing I had to do was figure out how to use a Jabber client to send and receive messages with Twitter. For some reason my Google-Fu failed me and figuring out how to do that took a lot of time. Check out the post I created to get you through the task of hooking up Twitter with Jabber.
Update: It seems that there is some confusion that each Twitter user would need to pair their Twitter account with a Jabber ID. Not necessary. We only need to do this for the Dovetail Twitter Integration Bot’s Twitter account. Twitter users, um… just use Twitter.
Next I wanted to switch my Twitter integration to use Jabber rather than the Twitter API. This turned out to be pretty easy and I am here to share what I learned along the way.
Add 1 cup of Jabber library
I went looking for .Net libraries that support Jabber/XMPP and found two: jabber-net and agsXMPP. I started off using jabber-net just because the license seemed more compatible with my goals (IANAL).
Using the jabber-net library is easy. Their examples are very drag-n-drop GUI driven but there is a console application example in their source. My main complaint is that I wish there were better testing seams in place. There are few interfaces and the JabberClient feels a bit like a god class. Enough complaining. On the upside for what we need to do it works. You basically use JabberClient to Connect to the Jabber server and add a OnMessage event handler that does the heavy lifting.
Moving from polling to a push architecture
Moving to Jabber allows us to benefit from a push architecture. We are basically asking Twitter to push to us an instant message every time our account receives a direct message. This is much better than polling the Twitter API continually asking their API, "Any messages? … How about now? … Now?". A push is lower impact on the Twitter servers and we receive the message almost instantaneously. The main downside is that you need to maintain a constant network connection between Jabber client and server.
All my communication with Twitter goes through a TwitterAgent. To move from the HTTP APIs to Jabber I needed to rework my TwitterAgent class which looks like this:
public interface IHttpTwitterAgent
{
IEnumerableIMessage GetMessages();
void SendDirectMessage(IMessage message);
void DeleteMessage(IMessage message);
}
To look a little something like this:
public interface IJabberTwitterAgent
{
void Connect();
void Disconnect();
event ActionIMessage OnDirectMessageReceived;
void SendDirectMessage(IMessage message);
}
I was able to quickly rework my Twitter Bot to use Jabber because I have a well defined interface between my application and Twitter. The main thing that changed was that getting direct messages now involves an event that is fired. In reality, this greatly simplified things as I no longer need code to poll the TwitterAgent for messages.
How it works
My implementation of the JabberTwitterAgent basically connects to the Jabber server using the credentials of a Jabber ID that is paired with the Dovetail Twitter account. When direct messages are received via Jabber a little bit of parsing needs to happen to parse out the sender’s Twitter name and message body. Here is an example message:
Direct from KevM:
close case 1234 Called the customer and told them to reboot the serverReply with ‘d KevM hi.’
The message is parsed out and sent to the existing messaging pipeline which further takes the message apart to determine what to do with it. In the case above the Dovetail case 1234 would get closed with a note of ‘Called the customer and told them to reboot the server’.
Responses are sent out using the JabberTwitterAgent with very little difference from the previous version. They just need to be formatted using the Twitter direct message syntax:
d <receiver> Message Text
Benefits
After the move to a Jabber based agent. I am seeing immediate reception of direct messages. Also, I no longer need to delete direct messages to prevent them from continually being returned by requests for direct messages further improving stability as the delete message Twitter API would sometimes not work.
Problems
Connection Stability
Currently I am trying to bullet proof maintenance of the connection between the Jabber client and server. Currently if the network connection goes down for some reason I do not always get reconnected to the server.
Server Certificates
The Jabber80 service supports secure communications but does not have a trusted certification authority. I needed to add a handler for the OnInvalidCertificate event.
private bool jc_OnInvalidCertificate(object sender, X509Certificate certificate, X509Chain
chain, SslPolicyErrors sslPolicyErrors)
{
_logger.DebugFormat("Invalid certificate ({0}):n{1}", sslPolicyErrors, certificate.ToString(true));
return true;
}
What’s in a Name
I am not excited about calling what I am working on Dovetail Twitter Integration. Calling it a Dovetail Twitter Mashup would be much cooler but this is technically not a mashup as I am not, yet, overlaying Twitter functionality in our Dovetail Agent web application. Anyone have a better suggestion for a project title?
