adrift in the sea of experience

Sunday, February 24, 2008

Logging all your chats in one place

I currently use used to use pidgin for chatting, mainly because it is cross-platform and I liked using the same application on windows and linux. Another advantage of pidgin was that it is designed to support almost all the chat protocols out there. I can be online on MSN and google talk at the same time with the same application.



One thing has been bugging me for a while now though, and that's the fact that my pidgin chat logs are stored locally. "Locally" turns out to mean "spread out over several machines and user accounts", without any hope of making them searchable or properly backed up. This is not a good thing. Fortunately this is only a problem for MSN chats. It doesn't matter for my google talk account, as the google talk server will log everything for me "in the cloud" and even make it searchable from the gmail web interface.



Is there anyway to get this advantage for other chat accounts also? Well actually, it turns out that there is. Google talk is just an jabber/XMPP implementation, and google has enabled "federation" with other jabber servers. That means that anyone can run their own jabber server, create users for it, and chat with google chat users. The jabber spec also includes the concept of "transports", which are basically gateway services running on the jabber service which enable you to talk to other networks like MSN. All you need to do is register an MSN transport with your google talk account, do all your MSN chatting via your google account, and voila: all your MSN chats are stored at google.

Here's how to do it:
1) Google talk or pidgin doesn't enable you to add transports. Download and install psi instead, another jabber client.
2) Psi will prompt you to add a jabber account when you first start it. Use your gmail address as a jabber-id. Set the status to "online", you should be connected to google talk now.
3) Right click on gmail.com in psi and click "service discovery". Psi will automatically look for services at "gmail.com" and show an error message because there aren't any.
4) Enter another jabber server in the service discovery window, one which runs an MSN transport (e.g. jabber.belnet.be) or some other transport for your favorite chat network.
5) Locate the MSN transport in the list of services, right-click it and click "register". This will prompt you for your MSN credentials.

6) Add/Authorize the transport contact which now appears in your main psi window (the "contact roster"). Double click it and allow it to subscribe to your presence.
7) All your MSN contacts will now also appear, each one needs to be accepted for addition in your contact list. (It may be faster to close psi at this point, log in with pidgin and accept the contacts from there.)
8) Say something to one of your MSN buddies
9) Go to gmail.com, log in, click "chats" or search for what you just said. It's all there! :) All your MSN chats are now logged by google, as long as you use your name@gmail.com jabber account for chatting.

Thursday, February 21, 2008

Finding debian packages and listing the files installed by them



The great thing about GNU/linux distributions is that they come with zillions of software packages, downloadable over the internet from free software repositories. Say you notice that your firefox browser doesn't support java applets. In debian, you're just a

apt-cache search java mozilla

away from finding a package providing just what you need. One of the results returned on my machine by the above command was "sun-java6-plugin". Just do

apt-get install sun-java6-plugin

and you're done! No need to worry about dependencies (like, ehm, java!), those are also downloaded and installed as needed.

Using an installed package

When you install an application "foo", typically it will be in a package named "foo" and will be launchable by typing "foo" in the console. In the case of command line tools, typing "man foo" will bring up one of those fascinating manual pages. In the case of GUI tools, they should be automagically added to your start menu in your favorite window manager. So far so good.

Unfortunately, a minority of the packages are a bit evil in this aspect. The "bittorrent" package for example provides the original bittorrent implementation as a set of command line tools. You can install it, but you'll find that no "bittorrent" command becomes available, nor will "man bittorrent" give any results. "apt-cache show bittorrent" doesn't provide much hints either. Where the hell are my command line tools?!

There is a solution! It took me a while to find, but it is to do a

dpkg -S bittorrent|grep bin

which will list all the invokable binaries installed by the bittorrent package. Turns out that the tool is invokable as btdownloadcurses.bittorrent (thank you tab completion!).

Getting started with a new programming library

This is also very handy for other types of packages for which it may not be obvious how to start using them, like programming libraries. Suppose you want to write a little script to upload your pictures to google's picasaweb service. A quick

apt-cache search picasa

will yield the "python-gdata" package. Once installed,

dpkg -S python-gdata

will help you locate all the new python goodness at your disposal. In this case, it turns out that alot of sample scripts are installed at /usr/share/doc/python-gdata to help you get started.

Monday, February 4, 2008

Recording phone calls on a S60 phone. Part 3: getting rid of the beep

OK so recording phone calls turned out to be relatively easy, but what about that annoying beep inserted into recorded phone conversations? This beep comes after the first second of the conversation, and every 15 seconds after that. Turns out that there is a way around this: the trick to avoid the beep is to not record longer than 1 second.

That's a pretty short conversation, you say?

Not necessarily. You could, for example, stop the recording before the first second completes, then restart it immediately, etc... Unfortunately the speed at which code can be executed is limited, and this technique results in unacceptable gaps in the recording, especially if we use python. There is a commercial application called Total Recall which appears to use this technique. It worked for my N95 if I set the recording quality to "low", but like my python script, the recording gaps are unacceptable.

Is it possible to do better? Yes! If you start a new recording before stopping the previous one, the beep still doesn't surface, and there will be no lost audio. The following function does exactly that:


def start_recording(filename):
global recording
counter = 0
recording = True
sound_file = audio.Sound.open(filename+'_part'+str(counter)+'.wav')
sound_file.record()
start = True
while recording:
if start:
start = False
else:
e32.ao_sleep(.6)
counter = counter + 1
next_sound_file = audio.Sound.open(filename+'_part'+str(counter)+'.wav')
next_sound_file.record()
e32.ao_sleep(.3)
sound_file.stop()
sound_file.close()
sound_file = next_sound_file
print "stopping recording"
sound_file.stop()
sound_file.close()
concatenate_wavs(filename, counter)


This function will create a recording in fragments of less than 1 second, which overlap about .3 seconds. In practice, it turns out that the overlap is typically a bit less than that. Now we could post-process these audio fragments on a PC to stitch them together, but part of the usefullness of recording phone conversations is that you can immediately replay them on your phone. This implies the need to do the post-processing on the phone itself.

This is were the python wave module comes in. This module provides all the functionality you need to read and write valid .wav files, but unfortunately it is not included in the PyS60 implementation. Not one to be easily discouraged, I downloaded the official python 2.2 sources. This is the (old!) version upon which PyS60 is based. Unpack the source archive, copy the wave.py and chunk.py modules to the phone, and yes, it works!

Simply concatenating the wave samples will result in a little stutter in the recording though. The following function attempts to detect the overlap between fragments to fix this:


def concatenate_wavs(filename, max):
print "Processing "+str(max+1)+" audio chunks:"
merged = wave.open(filename+'.wav','w')
previousframes = None
for i in range(0,max+1):
partname = filename+'_part'+str(i)+'.wav'
part = wave.open(partname,'r')
frames = part.readframes(part.getnframes())
part.close()
os.remove(partname)
if i==0:
merged.setparams(part.getparams())
overlap=0
else:
overlap=frames.find(previousframes[-8:])+8
if overlap==7:
overlap=0
print overlap
previousframes=frames
merged.writeframes(frames[overlap:])

merged.close()
print "done!"


And there you go, a simple python script for recording phone conversations on a S60 3rd edition phone without beeps :) For some reason the overlap detection does not always work, but the casual stutter is not yet annoying enough to warrant further investigation. More important is the fact that this script still needs alot of work before it can be declared suitable for public consumption. It should get a proper GUI, be packaged as a standalone application which can be easily installed, and it should be started automatically whenever the phone is booted.

But I have not yet figured out all that stuff, so that's for another post... :)