POuL

OCD: monitoring the Kimsufi webpages — the nerd way!

October 03, 2013 | 8 Minute Read

I do want a Kimsufi. But they’re in sold-out status since a month (about since someone made HackerNews notice they were renting dedicated servers for as low as 2,99€/month).

Well, that means that in order to get my own Kimsufi, I’ll have to smash competitors on time. That is, get to know when Kimsufis will be made available again before anyone else know and/or order one before anyone else.

This approach can be summarized into two steps.

  1. Bring my laptop and my cellphone (for wifi tethering) always with me.
  2. Write some sort of program that monitors the OVH website on my behalf
  3. Get notified as soon as something changes in the monitored pages
  4. When Kimsufis are made available again, get one

Step one it’s easy, as I almost always have my laptop and cell phone with me (when I’m not home I’m in university).

Step two and three are somehow tricky, but I solved it.

Things I most frequently check are SMS on my cellphone, as many friends of mine tend to use SMS service and in I tend to check emails on my laptop and not on my cellphone.

So the best way to get notice is SMS, at least for me.

How do I send SMS from a computer? Several options are available:

  1. some websites let you send SMS for free
  2. some website sell sms-sending services
  3. Skype let’s you send sms

Here are my considerations on the options:

  1. those website typically use CAPTCHA, and this makes it difficult to mechanize the process. I’m not (yet) capable of write captcha-decoding programs, so I have to turn down this option
  2. I didn’t actually do too much research on this, but buying a 100-sms package when I’m probably going to use something like 10-15 sms is probably a waste. It’s like killing a fly with a bazooka.
  3. Skype let’s me send sms using my credit (and I bought something like 10€ of skype credit two years ago when I needed to make calls in the UK). Skype is scriptable, and this lead me to choose Skype.

Scripting Skype is easy using Skype4Py library.

How to continuously check OVH webpages? This is a typical watchdog problem.

Conceptually speaking, it simple:

  1. Download $page and save to $old_page
  2. Sleep $wait_time
  3. Download $page againd and save to $new_page
  4. Do $new_page and $old_page differ ?
    If yes, go to step 5, otherwise go to step 6.
  5. (action triggered) Notify me
  6. Save $new_page to $old_page.
  7. Go to step 2

Writing a quick&dirty solution that in Perl would be simple using WWW::Mechanize, but I wanted to monitor more that one page at the same time, notify to different people (a friend of mine wanted  in Kimsufi too) and by the way I wanted to pratice object-oriented programming in Perl using Moose and experiment multi-threaded programming using Perl.

So, what did I do ?

First of all, I wrote a simple Python (using the aforementioned Skype4Py library) script to send SMS from the command line, called skype-sms-shooter. It takes only two parameters –body and –recipient, definitely self-explaining. Body must be shorter than 160 characters in length.

Then I proceeded in writing a watchdog program in Perl, maily made of two classes: Watchdog::Page and Watchdog::ThreadPool.

Watchdog::Page represent a page being monitored, stores its data and can invoke a function when action is triggered (change detected) and it can be specified by code (that is, you can tell to run your own code when change is detected) and pass parameters too.

My function was, of course, a piece of code that logs to console and runs skype-sms-shooter with appropriate parameters.

Full sources for this project are on my github page, in the ovh-watchdog repository.

Writing the code has been quite fun.

But here comes the really funny part: getting it to run 24/24h.

This isn’t a simple script that I can run in screen as I do typically.

Due to the skype instance running, we need some sort of X server running.

Does this mean I’ll have to run it on my laptop? No, that wouldn’t make that much sense as it would stop working when I suspend my laptop to ram (I rarely turn it off). Instead, this simply mean I have to run it on a computer which is 24/24h turned on. Like my Linode server or on dobby, my home server.

I’ve choosen to make it run on my Linode instance because the set-up process let me do some interesting things.

Of course, I won’t be running XOrg on my Linode. That requires a graphic card and a monitor, and I have none of them attached to my Linode.

Instead, I can use Xvfb, an headless X-compatible display server.

Ok, what’s the plan ?

  1. Set up a graphical environment using Xvfb
  2. Install skype
  3. Download ovh-watchdog
  4. Set up things
  5. run ovh_mehcanize.pl

Step one is easy, example from the Wikipedia page i’ve linked for Xvfb contain most of the things I needed. Additional information can be found in the Xvfb man page.

export DISPLAY=:1

screen -S xvfb

This is how I set up a screen instance for Xvfb. I like using different instances of gnu screen for different topics. I have a “main” instance for irc client and mutt, and an “xvfb” instance for Xvfb.

Now, inside screen (you can attach to it using screen -r xvfb)

Xvfb :1 -screen 0 1366x768x16 ## start xvfb &

fluxbox &

x11vnc -display :1 -bg -nopw -listen localhost -xkb

That is:

I start an X server and let it go doing its things in the background.

Then I start fluxbox, which provides a basic and light yet comfortable desktop environment.

Then I start x11vnc, a simple vnc server so that I can control the graphic enviroment.

Now: you may have noticed that I used the -nopw arguments, that tells x11vnc anyone to log in and see/use what’s happening on my desktop.

This is NOT a safe choice at all, but works in case the firewall is configured not to allow connection on port 5900 (standard vnc port).

In my case I have a password for vnc, and my firewall is configured not to allow connection on port 5900. I use ssh tunnelling to reach my vnc server.

On our machine (at home) now I can type:

ssh @santoro.tk -T -L 5900:localhost:5900

This creates an ssh tunnel, letting us reach localhost:5900 on my server.

Actually, on my server I run Xfce4, here is a screenshot:

did i tell you linode is great ?

did i tell you linode is great ?

In the image, skype is running in the Xvfb on my linode instance.

How to properly set up mechanize_ovh.pl ?

Simply open it into a text editor, reach the last part of the file, find part where pages are declared:

my $pag_k24g = Watchdog::Page->new(url => “http://www.ovh.it/server_dedicati/kimsufi_24g.xml”,
title => “Kimsufi 24g”,
change_detected => \&notify_sms,
change_detected_params => ["+39329xxxxxxx", "+39320xxxxxx"]) ;

in change_detected_params put your cell phone number, with international prefix.

I suggest you to try a custom page first, as Skype will ask for permission to let an application use its messaging service. Use a custom page, change it and let ovh_mechanize.pl recognize the change and allow programs to use Skype’s messaging capabilities.

You can now add al the pages you want.

Now I’m realizing I have to write a SETUP file for ovh-watchdog and a tl;dr section for this post.

Enjoy!