Tuesday, June 14, 2011

Replacing Lightweight Web Services with Twitter Bots!

Foresight
Here is a vague idea. I'm not sure if i can even call it an idea. I always had the tendency to create lightweight information portals as usable web-based services with a very simple user interface. It just used to get things done and nothing more.

Practical Use Case
For example, let us consider a simple problem of "PNR Status Enquiry" in Indian Trains (for the uninitiated: PNR status is nothing but the current status of a waitlisted ticket in Indian Trains). First of all, such a system is really useful because the official Indian Railways websites are a bit clogged and they don't provide any alert services. Also, their websites are not so catchy and mobile compatible, and we definitely don't want to switch on the computer just to check the PNR status.

Before today, if i was asked to build such a system, i would go for an elegant google-like web page, where there is just a text box and a button for the user to enter the PNR number and click go. The resultant page will be again a simple HTML table with the ticket details (of course all of them scrapped from one of the railway websites - pretty sure that this isn't legal, although i am not aware of any laws against site scraping).

Do we really need web for this?
But this thought stuck me today. Web is a wonderful platform, at times too good to host silly and redundant stuff like this. Tomorrow i may have to build a similar system for Buses or Aeroplanes requiring me to add more and more pages with almost same functionality but different information sources. We don't need the web for such silly things. My idea is that, why not use "Twitter" as a platform for such web services. I've heard of facebook as a platform before for many applications and games (Farmville), so why not twitter as a platform too?

Twitter as a Platform
So what exactly do i mean by twitter as a platform? Let us redesign the same PNR status enquiry system using what i mean by twitter as a platform. Instead of having a web page for inputs, lets host a twitter bot, say @pnrbot. Now, whenever you need to enquire the status of a PNR number, all you have to do is post a tweet mentioning that bot, for example "@pnrbot 1234567890" (where 1234567890 is the PNR number you wish to enquire).

Now, as long as your tweets are public, the bot can read your tweet almost instantaneously thanks to the vast amount of real time APIs provided by twitter. Now the bot does the usual site scraping from the railways website for your PNR status and it posts it as a reply to your tweet. Simple isn't it? To take it a level further, the bot can also autotweet your PNR status every day until your journey date, which is not very easy in the case of a web based app. Also, since you will not be following the bot and the bot will not be following you, you will not clutter any of your friends' timeline with this tweet as it won't appear in their timeline.

If any of the input data is sensitive and not to be exposed, then the same design can be adopted by just switching the term "tweet" with "Direct Message" (although in that case, both you and the bot need to be following each other, which can be easily accomplished).

Pros of the Platform
Twitter is a part of our day to day life since the inception of mobile internet. So, you get many useful information from such bots interactively rather than opening your webpage and waiting for the page to load. Moreover, twitter is known for its notifications. If you have activated SMS alerts for @ mentions, then you don't even need to have internet to make use of such a bot. You can just send the tweet through an SMS and read the reply from the bot as SMS. Also, twitter has email notifications which may also be of good use. As mentioned in the example, you can have one input with multiple periodical outputs (like time based notifications, etc.) which is not so easy to implement in web based systems.

Developer Standpoint
So from a developer standpoint, what do we ultimately gain by choosing Twitter as a platform over Web for lightweight services? The answer is quite simple and really advantageous. For one, you need not host a web server for lightweight web services. All you need to do is run a script that will act as the bot. The script will also be really lightweight since twitter APIs does all the pushing for you (no polling). You offload majority of the user interface and load to twitter and do only the actual processing in your server.

From the implementation perspective, it'd be really great to have a good library/framework in a nice scripting language (like php or python) for building such a bot so that the possible features (like twitter API access, etc.) could be abstracted out thereby the developers actually have to write just the logic of their actual bot and nothing extra.

Closing Thoughts
I am pretty sure such bots already exists. By building more and more of interactive bots Twitter can really stand tall as a good platform not just for communication but also for instantaneous information retrieval.

P.S.: I am in the process of developing a simple such bot as a proof-of-concept. I am also highly determined to come up with a generic twitter bot library as i mentioned in the post. So as always, interested developers are welcome to carry on if you like the idea!

-Vignesh

8 comments:

  1. Hey Vig,
    Nice initiative! Hats off to your 'usual' different way of approaching a real time problem to make life simple!!!

    ReplyDelete
  2. Sweet! Looks good man. As for the idea I think its neat. Go for it!
    Here are somethings that you may want to consider

    1) Twitter did have some limitations on their API. There is frequent downtimes. I face a frequent downtime especially with user profile images. This may not be particularly applicable. However, consider this since you will have to keep polling looking for tweets. Dont think they have any push callback api yet.

    This uptime is something that you need to watch out for since your looking to serve time critical needs like PNR.

    2) Try and see if the API can be language agnostic. Let it not be a specific PHP lib or Python Lib. Try and see if you can get it to work on any language on a POSIX system. I am trying something like this myself. I am a big fan of Firefox's intermediate Language from C++ to XUL and js.

    Personally, the idea is neat. Dont limit it to being used by a single language.
    http://www.imagemagick.org/script/api.php
    is a good example of multiple prog interfaces.

    ReplyDelete
  3. Hi na,

    First of all thanks a lot for your feedback. Regarding implementation, i am not seriously into anything yet. But will surely consider and try building it into a language independent library.

    Regarding twitter API, they do have a push based API (they call it streaming API, it works by opening a persistent socket connection with twitter servers), so there is no polling. Regarding downtime of twitter, its really a calculated trade off between criticality of the application in question vs uptime of an equivalent self hosted web based server. I personally think twitter will definitely win over the self hosted solution.

    ReplyDelete
  4. NP! Interesting you should say that Twitter's uptime is better than your own hosted application.
    I would have nothing valid if this was a google-app or EC2. But twitter and linked in have the most controversial API. Down times are not as rare as you think. Down time of API != Downtime of twitter. Its far more frequent in my experience.
    That said, I have not used the streaming API that you have just mentioned.

    ReplyDelete
  5. Ah, Downtime of the API != Downtime of twitter. I didn't exactly realize this point. May be true for critical services, but a vast number of lightweight web applications that aren't really critical can definitely work better in this form of interface.

    ReplyDelete
  6. I like this idea. Brilliant. All the best for implementation.

    Few points:
    - There are certain such bots designed to do some custom operations on a IRC channel (instead of the twitter) (like looking up phone numbers, taking meeting minutes, etc. search for supybot)
    - This may put up some extra load on the IRCTC site contributing to the slowness of the servers
    - The whole IRCTC thing needs to be revamped. Screenscrapping is unsafe for your battery life, memory cycles etc. But it is not something under our control though ;)
    - You can probably ask for IRCTC to ask for notifications / API support which they can charge.
    - I am not sure how long twitter may support having such an app. / bot as they seem to not benefit out of it. Integrating such a bot with Outlook/Exchange/GMail may be more efficient and safe.

    ReplyDelete
  7. U've used this WC cricket scores that Raghav did scrapin data from cricinfo and tweeting it, to whose feeds u can subscribe from mobile?

    ReplyDelete
  8. Nice idea dude.. do one stuff..will be really useful :)

    ReplyDelete