Tags: application, calendar, google calender, helpdesk, ical, nagios, python, shifts
Comments Off
Kumishifts: The Release
[Repost from Kumiblog]
One of my personal missions within Kumina is to decrease the amount of noise. We work fairly event-driven, responding to everything that gets reported. Although I believe our customers like this very much, it can be a bit of a bother in the case where someone is not on helpdesk duty. One of my pet peeves was the fact that Nagios sent SMS alerts to everyone, instead of the person who was on duty at the time.
Since we have multiple Nagios instances, it would be a bit of a hassle to change the config everywhere all the time, so we stuck with the current way. At least you can be sure that someone received the message. But since we get some false positives too, at time, it does add to the bill.
So I decided to create Kumishifts. This little Python script takes a Google Calendar (or any ical you point it at) and distills who’s on duty. It generated a Nagios contacts file based on that information (and then some). We can now actually work with escalations to make sure that if the first person responsible doesn’t respond fast enough (bad person!), a second one will get the message after a little while, too.
We’re not yet deploying the script everywhere, but will soon, after it’s properly tested. I’d appreciated any feedback on the app!
[FunkLoad] Random tests for benchmarking
I’ve recently started using FunkLoad to do benchmarking of websites. It works quite okay and I love the fact that it’s Python, since that’s my language of choice.
One thing that kept bugging me was that since FunkLoad is more a unittester, it can by default only run a single run multiple times against a site. That won’t do. You want several different kinds of users visiting your website when you’re benchmarking. But since everything is Python, I was able to solve that like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 | # -*- coding: iso-8859-15 -*- """loadtest FunkLoad test $Id: $ """ import unittest, random, sys from funkload.FunkLoadTestCase import FunkLoadTestCase from webunit.utility import Upload from funkload.utils import Data #from funkload.utils import xmlrpc_get_credential class LoadTest(FunkLoadTestCase): """ Loadtest for website. This test use a configuration file LoadTest.conf. """ def setUp(self): """Setting up test.""" self.logd("setUp") self.server_url = self.conf_get('main', 'url') def doTest(self): # Picks a test at random tests = ["8081","8082","8083","8084","8085","8086"] rnd = random.choice(tests) return getattr(self, "test_"+str(rnd))() def test_8081(self): # The description should be set in the configuration file server_url = self.server_url # begin of test --------------------------------------------- # end of test--------------------------------------------- def test_8082(self): # The description should be set in the configuration file server_url = self.server_url # begin of test --------------------------------------------- # end of test--------------------------------------------- |
(I left out the test themselves, but I’m sure you can find where to place them.)
The recorded tests are in the same class, and they’re called test_8081, test_8082, etc. up to test_8086. These corresponded to the proxy ports on which fl-record was listening when the session was recorded. Line 27 makes sure that the test that was chosen gets run.
You can of course expand this as much as you like, for example adding weights for certain tests. Kinda neat, python.
Python: My .vimrc
This is especially for working with Python. I welcome additions or better practises!
set term=builtin_ansi syntax on autocmd BufRead *.py set smartindent cinwords=if,elif,else,for,while,try,except,finally,def,class autocmd BufRead *.py inoremap # X^H# autocmd BufRead *.py set tabstop=4 autocmd BufRead *.py set shiftwidth=4 autocmd BufRead *.py set smarttab autocmd BufRead *.py set expandtab autocmd BufRead *.py set softtabstop=4 autocmd BufRead *.py set autoindent autocmd BufRead *.py highlight BadWhitespace ctermbg=red guibg=red autocmd BufRead *.py match BadWhitespace /^\t\+/ autocmd BufRead *.py match BadWhitespace /\s\+$/
(The ^H is typed by Ctrl+V Ctrl+H.)
Python: IMAP IDLE with imaplib2
Had a little fun today trying to get IMAP IDLE from Python working. It’s not in the default imaplib, but Piers Lauder’s imaplib2 has support for it. (I read on another blog that Piers Lauder is also the writer of imaplib, so that’s kinda nice.)
I don’t really program a lot anymore, so I had some trouble trying to understand how it’s supposed to work. Luckily, the blog post that I linked above, also had a link to the git sources of offlineimap, which uses it. Their imapserver.py has a nice implementation that I kind of copied.
As a proof of concept, and hopefully so others can use this too, I created the script below to see if I could get it to work. It doesn’t do a whole lot, just print a line when the mailbox gets a new mail. But you can build on it from there. I need these kind of examples to get stuff like this, so I hope it helps someone else! Leave me a comment if it does.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | import imaplib2, time from threading import * # This is the threading object that does all the waiting on # the event class Idler(object): def __init__(self, conn): self.thread = Thread(target=self.idle) self.M = conn self.event = Event() def start(self): self.thread.start() def stop(self): # This is a neat trick to make thread end. Took me a # while to figure that one out! self.event.set() def join(self): self.thread.join() def idle(self): # Starting an unending loop here while True: # This is part of the trick to make the loop stop # when the stop() command is given if self.event.isSet(): return self.needsync = False # A callback method that gets called when a new # email arrives. Very basic, but that's good. def callback(args): if not self.event.isSet(): self.needsync = True self.event.set() # Do the actual idle call. This returns immediately, # since it's asynchronous. self.M.idle(callback=callback) # This waits until the event is set. The event is # set by the callback, when the server 'answers' # the idle call and the callback function gets # called. self.event.wait() # Because the function sets the needsync variable, # this helps escape the loop without doing # anything if the stop() is called. Kinda neat # solution. if self.needsync: self.event.clear() self.dosync() # The method that gets called when a new email arrives. # Replace it with something better. def dosync(self): print "Got an event!" # Had to do this stuff in a try-finally, since some testing # went a little wrong..... try: # Set the following two lines to your creds and server M = imaplib2.IMAP4_SSL("mail.example.com") M.login("mylogin","mypassword") # We need to get out of the AUTH state, so we just select # the INBOX. M.select("INBOX") # Start the Idler thread idler = Idler(M) idler.start() # Because this is just an example, exit after 1 minute. time.sleep(1*60) finally: # Clean up. idler.stop() idler.join() M.close() # This is important! M.logout() |
Python: Reverse reader
Sometimes you want to read files from the end to the front. You can find a lot of recipes online, but I’m using the one in the comments from here (scroll down). Bonus feature, I can use that one as an iterator, so csv.reader and other stuff like it can use it. Blogging it here, so I know where to find it again if I need it 
4 Years of Blogging
So I timed this post at exactly four years after I wrote my first post on this blog. A lot has changed since then. Maybe I should re-read some of those older messages.
Hehehe, I was still using Mandriva (then known as Mandrake) back then. And even then I was working on joining RSS feeds and bayesian filters! How cool is that? I keep coming back to that project and starting it anew every year, I guess. Hope I can get it to work one of these days. I still think the idea is great, if only I could get myself to spend a few days on the project, I know I could make it work. Ah well…
duplicity in MacPorts, backup to S3
Duplicity from MacPorts is currently broken. The problem is in the portfile, which wants to use Python 2.5. However, there is no OpenSSL package for Python 2.5 in MacPorts, which gives you this error if you try to backup to S3 through SSL:
File "/opt/local/lib/python2.5/httplib.py", line 1135, in connect
ssl = socket.ssl(sock, self.key_file, self.cert_file)
AttributeError: 'module' object has no attribute 'ssl'
The solution is to edit the Portfile (which lives in /opt/local/var/macports/sources/rsync.macports.org/release/ports/sysutils/duplicity/Portfile on my machine) and change “python25″ in “python24″ and halfway through the file change the py25-* packages in py-* packages. Then install the port. That fixes it.
Don’t forget to *not* upgrade the port without changing the Portfile…
Isoformat in Python 2.4
Sometimes I don’t like python. Just sometimes, I love it most of the times. But just look what hoops I need to jump through to get a current timestamp in ISO format:
from datetime import datetime, tzinfo, timedelta
import time
class TZ(tzinfo):
def utcoffset(self,dt): return timedelta(seconds=time.timezone)
def dst(self,dt): return timedelta(0)
a = datetime.now(TZ())
a= a.replace(microsecond = 0)
print a.isoformat()
That prints:
asterisk:~ tim$ python test.py
2008-03-24T13:38:06-01:00
Fugly…
PyQt4 on Windows
Yes, you read that correctly, on Windows. Wanted to make a .exe out of an app I’m working on, so I had to install PyQt4 on Windows. I never use Windows, so I was at a loss. But for those others who are looking for it:
You can change the env vars like %PATH% in Configuration screen > System > Advanced.
Finding that took me about 45 minutes. God, I hate Windows.
PyQt4 on MacOSX with Macports
Yesterday evening, I created a working py-pyqt4 Portfile for Macports. At least, it worked for me. You can find it in ticket #14744 on the Macports Trac. Hope this helps someone.
Fallen in love again…
… with programming (if you expected “my girlfriend” here, I’ve never stopped loving her). I’ve been going through the Django tutorial and I’m really liking doing webdevelopment again. Yes, I know, I was working with Ruby on Rails earlier, but Django is much more to my liking. I have to quit now with the tutorial because I’m sleepy, but I will continue with it tomorrow. Take a look at it… It’s really, really cool.
On a sidenote, maybe if Ruby on Rails had such a cool tutorial, I would fall for that language, but I’m struggling much more with Ruby on Rails. And I already know (some) Python, which makes Django much easier for me. What are you waiting for?? Go take a look!
