Monday, June 30, 2008

Sending Binary files over xml rpc

I'm posting this because I finally figured how to send files(pictures in particular) over xmlrpc. It took me a while to figure it out since I was mostly concerned with saving the files into database but here it is.


#!/usr/bin/env python

from twisted.web import xmlrpc, server
import os

class PositRPC(xmlrpc.XMLRPC):
        def xmlrpc_savefile(self, file):
                datum = file.data
                thefile = open('/tmp/gnu.jpg', "wb")
                #filebuffer = file
                thefile.write(datum)
                thefile.close()
                #os.spawnl(os.P_NOWAIT, '/usr/bin/display', '/tmp/gnu.jpg')  
                print datum
                return 1

if __name__ == '__main__':
    from twisted.internet import reactor
    r = PositRPC()
    reactor.listenTCP(7080, server.Site(r))
    reactor.run()




Then I opened trusty IDLE
>>> import xmlrpclib
>>> s = xmlrpclib.Server('http://127.0.0.1:7080')
>>> d = xmlrpclib.Binary(open('/home/pras/PicasaDocuments/gnu.jpg', 'rb'))
>>> s.savefile(d)


The idea here is that although the file is sent as a Binary, we need to convert it to string(bytes) to save it. Since we have opened the file with 'rb' option in IDLE and with 'wb' option in our server (respectively for read binary and write binary), we get binary file at the server end.

Sunday, June 29, 2008

Locking screen from command line

I use gnome and so far I've been happy with my Ctrl+Alt+L shortcut in gnome. It used to lock up the screen when I was away from the desktop. However, recently, I've started to use XFCE and sometimes ratpoison, which didn't have this shortcut.
Anyways, the simpler way to lock your screen is from command line and it seems to work everywhere I've tried so far.

$xscreensaver-command -lock
I just kept in here so that I don't forget. Maybe this little tip would be useful for some.

I just found an alternate method that can work too.
$gnome-screensaver
$gnome-screensaver-command -l

Friday, June 27, 2008

Code Highlighting in Blogger

Don't you just love syntax highlighting that we've become so used to these days? Be it the venerable emacs editor or the new and upcoming Notepad++es, we love our syntax highlighting editors. But the codes on our sites still do suffer this problem of being black and white. I solved this problem for our college's hfoss wiki by installing the GeSHi CodeTag Extension for mediawiki but while looking at my own posts in here, it seems the idea needs to be extended to blogger. It really looks too ugly to my used-to-pretty-IDE eyes.
In my quest to look for a solution, I hit this blog http://codeshepherd.blogspot.com/2007/09/syntax-highlighting-for-code-embedded.html
Which had two ideas. First one, to use vim's TOhtml. But being lazy to modify all my files over and over again, I decided to forego it. I copied the prettfy.js and prettify.css files to my ubuntu.org.np server and followed the instructions at http://google-code-prettify.googlecode.com/svn/trunk/README.html
The effects were, well, wierd. Apart from rather slow loading times that javascript induces, my code looked horrible with
s after every line. Totally horrible and waste of space. I looked at the code and zeroed in on line 834 where I found this regular expression:

833 var htmlChunk = PR_textToHtml(
834 tabExpander(sourceText.substring(outputIdx, sourceIdx)))
835 .replace(/(\r\n?|\n| ) /g, '$1 ')
836 .replace(/\r\n|\n/g, '
');

After playing around with this, breaking it and everything, I realized I'm not really helping anyone here. Apart from the fact that not everyone uses javascript by default, including me (thanks to noScript) and even I use some unconventional browsers from time to time just for kicks like w3m, elinks, lynx (Ahh, those good old days, when they were my only browsers). So, I went for the vim solution.

Now, the next frontier: Posting to blogger automagically from vim/emacs whichever is easier to automate from. (Yeah I know there's something for emacs to do this) but with this functionality right there for recognizing the code and htmlifying it.

I think I've had too much coffee tonight. I just realized, "Why am I posting this at 2AM when I have some nice programming to do?" Anyways, back to my programming. Hope this post helps someone. Or, if you know some tool for doing these things automatically, tell me so that I can save some time (or even some google time if that's what you wanted to say :).

Follow me on Twitter

After facebook, I haven't really got around to most of the social networking things, but after reading about it (a lot) and seeing some really famous people actively using it, I decided to give it a try.
Guess what, I'm loving it. So, if you are interested in following me, do follow me at
www.twitter.com/prasannagautam
Or if you want me to be notified of your tweets, do post in here so that I can give some thoughts to that :)

Implementing xmlrpc database server in twisted

A while back, for my POSIT project (for google android), I wrote a small application in python that sent out some data from the server's repository and saved to the database. I never really got around the saving part because I couldn't get the database to send pictures back and forth. In part, I went back to working on the phone itself and my studies because I was being a little too unfocused in necessary stuffs which was bugging me off. Yeah, that was during the semester.
Now, over the summer, I am getting to continue the same project and it's more fun with a better idea about what I want to do and especially how I need to do. Also, getting sick for a while helped because I wasn't really able to work much so I just read up a few books on software engineering. The best one undoubtedly is the Gang of Four's Design Patterns. I have yet to finish that one but truly good book indeed.
OK, enough ranting. You didn't come here to know about my reading patterns, did you? Ok, where was I? Yeah, I wrote the application but didn't really got through with it because I came up with some problems with transferring pictures, or blobs to and from the phone. So, this time around I decided to use the xmlrpc library from the twisted framework to the purpose.
First I created the server with the following script. I found it somewhere in the twistedmatrix site, (really excellent documentation btw).



#!/usr/bin/env python
from twisted.web import xmlrpc, server
#import DBconnect
import MySQLdb

class PositRPC(xmlrpc.XMLRPC):
"""An example object to be published."""

def xmlrpc_echo(self, x):
"""Return all passed args."""
return x

def xmlrpc_add(self, a, b):
"""Return sum of arguments."""
return a + b

def xmlrpc_getall(self):
"""Returns all the data from the sql server."""
db = MySQLdb.connect("127.0.0.1", "username", "password", "posit")
c= db.cursor(MySQLdb.cursors.DictCursor)
c.execute("SELECT id, latitude, longitude, description FROM db")
arr = []
result_set=c.fetchall()
for row in result_set:
#arr.append(xmlrpclib.Binary(row[0]))
#print row
entry = []
entry.append(row["id"])
entry.append(row["latitude"])
entry.append(row["longitude"])
entry.append(row["description"])
#entry.append(cPickle.dumps(row["photo"]))
arr.append(entry)
db.close()
return arr


def xmlrpc_savedb(self,latitude, longitude, description):
"""Save the data to the database"""
db = MySQLdb.connect("127.0.0.1", "username", "password", "posit")
c = db.cursor()
c.execute ("INSERT INTO db (latitude, longitude, description) VALUES ('"+latitude+"','"+longitude+"','"+description+"')")
db.close()
return "success"



if __name__ == '__main__':
from twisted.internet import reactor
r = PositRPC()
reactor.listenTCP(7080, server.Site(r))
reactor.run()



So, the idea here is to run the server on the 7080 port and pull some data with the getall function and save some datawith savedb function. Yeah, I really need to use some standard names. Heck, even I got confused about the names.
Also, see the success return for it? Yeah, it's there for a wierd little reason. Whenever I was sending data to the server using the savedb function, I was getting the following error:
Traceback (most recent call last):
File "", line 2, in ?
File "/usr/lib/python2.4/xmlrpclib.py", line 1096, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.4/xmlrpclib.py", line 1383, in __request
verbose=self.__verbose
File "/usr/lib/python2.4/xmlrpclib.py", line 1147, in request
return self._parse_response(h.getfile(), sock)
File "/usr/lib/python2.4/xmlrpclib.py", line 1286, in _parse_response
return u.close()
File "/usr/lib/python2.4/xmlrpclib.py", line 744, in close
raise Fault(**self._stack[0])
xmlrpclib.Fault:
First instict:Google the error message, which landed me to this page:
http://www.oreillynet.com/onlamp/blog/2005/10/why_i_ditched_xmlrpc_in_favor.html
Hmm, after searching for pyro and whatnot for a minute or two and deciding I'd rather go with interoperability that xmlrpc provides than go with a shiny toy that I need to figure how to use with java (oh yeah, that one would bite real hard later in my experience).
I tried allow_none but didn't work. So, I finally decided to just return something as a response from the server which worked. Turns out (this is mentioned in the oreilly link above) that there's no None in xmlrpc. So, I need to return one thing or other to avoid that nasty error. Hope this one helps someone save some time. :)

Oh yeah, you do need to have a database with that table.
Here's the SQL script for creating the table if you need it. (Oh, gosh you probably want to have known these itsy bitsy things about databases by now though)

CREATE TABLE IF NOT EXISTS `db` (
`id` int(11) NOT NULL auto_increment,
`latitude` double NOT NULL,
`longitude` double NOT NULL,
`description` text,
`photo` blob,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 COMMENT='the main database for everything' AUTO_INCREMENT=15 ;


Now, I open up the python console and type this up

>>>import xmlrpclib
>>>s=xmlrpclib.Server("http://scheme.cs.trincoll.edu:7080")
>>>s.getall()
[Returns me an array of arrays]
>>>s.savedb("1","2","description")

Anyhoo, I go back to my work now. Happy Hacking :)

Friday, June 20, 2008

Firefox 3 on debian etch

First, some updates:
These days, I've been busy doing my summer research (finally learning to document properly) and working part/full time for the computing center here as the system administrator (well, assisting the system administrator to be correct).
OK, the release of firefox 3 was big and I participated heavily and encouraged everyone to participate by downloading the browser within the 24 hours. So, now everyone I know of is running firefox 3 except the computers in my lab that I am managing. So, today after my research was over, I decided to look into the matter. There were a few problems that I was facing
1. I didn't want to kill of the old firefox
2. I didn't want to use what's considered to be unstable by debian etch in the core system
First of all, I got the error that firefox 3 needed gtk+2.10. I just downloaded the file from http://ftp.gnome.org/pub/gnome/sources/gtk+/2.10/gtk+-2.10.14.tar.bz2 and installed it at /opt/gtk

Here are the commands:
$ wget http://ftp.gnome.org/pub/gnome/sources/gtk+/2.10/gtk+-2.10.14.tar.bz2
$ tar -xvjf gtk+-2.10.14.tar.bz2
$ cd gtk+-2.10.14
$ ./configure --prefix=/opt/gtk
$ sudo su
# make && make install
# exit

Then I downloaded firefox from http://www.mozilla.com/products/download.html?product=firefox-3.0&os=linux&lang=en-US
and modified the run-mozilla.sh to make it run.
Here are the commands:
$ wget http://www.mozilla.com/products/download.html?product=firefox-3.0&os=linux&lang=en-US
$ sudo tar -xvf firefox-3.0.tar.gz -C /opt
This will give us a /opt/firefox folder
First of all, we need to make sure that firefox actually run by run-mozilla.sh file.
So, I added this for firefox path in the beginning (right after ****END LICENSE BLOCK****)
FF_PATH="/opt/firefox"
and changed MOZ_PROGRAM="" to MOZ_PROGRAM="$FF_PATH/firefox". Now firefox should run with the
Since all this is about gtk+ 2.10, I added
GLIB210_PATH="/opt/gtk/lib" ( I know it's a misnomer, I don't care about it at this point, you can change it to appropriate name)
and added it to the
LD_LIBRARY_PATH=${MOZ_DIST_BIN}:${MOZ_DIST_BIN}/plugins:${GLIB210_PATH}:${MRE_HOME}${LD_LIBRARY_PATH+":$LD_LIBRARY_PATH"}
in line 357.
Now firefox runs with ./run-mozilla.sh command.

But that's not enough, we want to have firefox run with a command, we want it on the menu along with iceweasel. So, I added firefox.desktop in /usr/share/applications
with this data:
[Desktop Entry]
Encoding=UTF-8
Name=Firefox 3 Web Browser
Exec=/opt/firefox/run-mozilla.sh %u
Comment= Firefox 3 - The latest and greatest from Mozilla
Terminal=false
X-MultipleArgs=false
Type=Application
Icon=firefox3.png
Categories=Application;Network;
MimeType=text/html;text/xml;application/xhtml+xml;application/xml;application/vnd.mozilla.xul+xml;application/rss+xml;application/rdf+xml;image/gif;image/jpeg;image/png
StartupNotify=true

Then I copied the file into /var/lib/menu-xdg/applications/X-Debian-Apps-Net-firefox.desktop
I copied the bundled icon with firefox from /opt/firefox/icons/mozicon128.png to /usr/share/pixmaps/firefox3.png
and voila, firefox-3 in the menu without upgrading the system's gtk. You could use it in your home directory if your system administrator doesn't upgrade it soon enough

Download
ftp://ftp.ubuntu.org.np/firefox-3-lab.tar.bz2
and do the following if you don't want to go through all that
$ wget ftp://ftp.ubuntu.org.np/firefox-3-lab.tar.bz2
This is for downloading, if you have downloaded already cd to that folder
$sudo tar -xjvf firefox-3-lab.tar.bz2 -C /
It saves the firefox 3 files in the /opt/firefox directory and adds icon to the Applications menu.
If you are on gnome do
$killall gnome-panel
wait a while for it to reload with your updated icons