Friday, June 27, 2008

Implementing xmlrpc database server in twisted

A while back, for my POSIT project (for google android), I wrote a small application in python that sent out some data from the server's repository and saved to the database. I never really got around the saving part because I couldn't get the database to send pictures back and forth. In part, I went back to working on the phone itself and my studies because I was being a little too unfocused in necessary stuffs which was bugging me off. Yeah, that was during the semester.
Now, over the summer, I am getting to continue the same project and it's more fun with a better idea about what I want to do and especially how I need to do. Also, getting sick for a while helped because I wasn't really able to work much so I just read up a few books on software engineering. The best one undoubtedly is the Gang of Four's Design Patterns. I have yet to finish that one but truly good book indeed.
OK, enough ranting. You didn't come here to know about my reading patterns, did you? Ok, where was I? Yeah, I wrote the application but didn't really got through with it because I came up with some problems with transferring pictures, or blobs to and from the phone. So, this time around I decided to use the xmlrpc library from the twisted framework to the purpose.
First I created the server with the following script. I found it somewhere in the twistedmatrix site, (really excellent documentation btw).

#!/usr/bin/env python
from twisted.web import xmlrpc, server
#import DBconnect
import MySQLdb

class PositRPC(xmlrpc.XMLRPC):
"""An example object to be published."""

def xmlrpc_echo(self, x):
"""Return all passed args."""
return x

def xmlrpc_add(self, a, b):
"""Return sum of arguments."""
return a + b

def xmlrpc_getall(self):
"""Returns all the data from the sql server."""
db = MySQLdb.connect("", "username", "password", "posit")
c= db.cursor(MySQLdb.cursors.DictCursor)
c.execute("SELECT id, latitude, longitude, description FROM db")
arr = []
for row in result_set:
#print row
entry = []
return arr

def xmlrpc_savedb(self,latitude, longitude, description):
"""Save the data to the database"""
db = MySQLdb.connect("", "username", "password", "posit")
c = db.cursor()
c.execute ("INSERT INTO db (latitude, longitude, description) VALUES ('"+latitude+"','"+longitude+"','"+description+"')")
return "success"

if __name__ == '__main__':
from twisted.internet import reactor
r = PositRPC()
reactor.listenTCP(7080, server.Site(r))

So, the idea here is to run the server on the 7080 port and pull some data with the getall function and save some datawith savedb function. Yeah, I really need to use some standard names. Heck, even I got confused about the names.
Also, see the success return for it? Yeah, it's there for a wierd little reason. Whenever I was sending data to the server using the savedb function, I was getting the following error:
Traceback (most recent call last):
File "", line 2, in ?
File "/usr/lib/python2.4/", line 1096, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.4/", line 1383, in __request
File "/usr/lib/python2.4/", line 1147, in request
return self._parse_response(h.getfile(), sock)
File "/usr/lib/python2.4/", line 1286, in _parse_response
return u.close()
File "/usr/lib/python2.4/", line 744, in close
raise Fault(**self._stack[0])
First instict:Google the error message, which landed me to this page:
Hmm, after searching for pyro and whatnot for a minute or two and deciding I'd rather go with interoperability that xmlrpc provides than go with a shiny toy that I need to figure how to use with java (oh yeah, that one would bite real hard later in my experience).
I tried allow_none but didn't work. So, I finally decided to just return something as a response from the server which worked. Turns out (this is mentioned in the oreilly link above) that there's no None in xmlrpc. So, I need to return one thing or other to avoid that nasty error. Hope this one helps someone save some time. :)

Oh yeah, you do need to have a database with that table.
Here's the SQL script for creating the table if you need it. (Oh, gosh you probably want to have known these itsy bitsy things about databases by now though)

`id` int(11) NOT NULL auto_increment,
`latitude` double NOT NULL,
`longitude` double NOT NULL,
`description` text,
`photo` blob,
) ENGINE=MyISAM DEFAULT CHARSET=latin1 COMMENT='the main database for everything' AUTO_INCREMENT=15 ;

Now, I open up the python console and type this up

>>>import xmlrpclib
[Returns me an array of arrays]

Anyhoo, I go back to my work now. Happy Hacking :)