Python Library for Google Search

You're viewing a comment by chad and its responses.

May 05, 2009, 00:59

Hi Peter, I was looking at your code. I have done a much simpler utility for searching google. I used to use beautifulsoup also, but now I use lxml and xpath. It produces much quicker and cleaner code... here is an example that returns an array of the urls and text:

from lxml import etree as et
from urllib import quote_plus,urlopen

def gsearch(q='',num=10,datelimit=''):  
        returninfo=[]
        searchurl='http://google.com/search?hl=en&as_q=%s&num=%s&as_qdr=%s'%(quote_plus(q),str(num),datelimit)
        results=urlopen(searchurl).read()
        tree=et.fromstring(results,et.HTMLParser())
        links=tree.xpath('/html/body[@id="gsr"]/div[@id="res"]/div[1]/ol/li/h3/a')
        for a in links:
            returninfo.append({'href':a.values()[0],'text':a.text})
        return returninfo

Let me know what you think!

Reply to this comment

Reply To This Comment

(why do I need your e-mail?)

(Your twitter name, if you have one. (I'm @pkrumins, btw.))

Comment Help

Type first 3 letters of your name: (just to make sure you're a human)

Please preview the comment before submitting to make sure it's OK.