Ip to Country for Pylons Comments

Saturday, 19 August 2006

As soon as I saw this post by Grégory Tappéro on planet python I couldn't wait to implement this in my Pylons powered blog.

(Note: To compare the effort needed on Pylons to implement this as compared to Django, I'll follow the same procedure as Gregory has followed, so that you can compare the steps 1-on-1.)

I'm implementing this on my home brewed blog software - Chiselman (unreleased). Please replace all references to chiselman with your proejects name.

What you need

• The ip-to-country data file, 60k entries as a csv file here.
• A set of tiny flags images, which you will put in /public/img/flags/ of your project.

Models

Create a iptocountry.py model file under chiselman/chiselman/models/ folder. Then we edit the models.py file, we need to make the fields match the ip-to-country.csv data file, and to add a method to import this data from cvs to your project database iptocountry table.

from sqlalchemy import *

dburi = 'mysql://user:passwd@localhost/chiselman'
db = create_engine(dburi)
metadata = BoundMetaData(db)

iptocountry = Table('iptocountry', metadata,
                    Column('id', Integer, primary_key=True),
                    Column('ip_from', Integer),
                    Column('ip_to', Integer),
                    Column('country_code2', String(2)),
                    Column('country_code3', String(3)),
                    Column('country_name', String(50))
                    )
class IPToCountry(object):
    def __repr__(self):
        return "%s %s %s" % (self.ip_from, self.ip_to, self.country_name)

def __init__(self, ipf=None, ipt=None, cc2=None, cc3=None, cname=None):
        if ipf and ipt and cc2 and cc3 and cname:
            self.ip_from = ipf
            self.ip_to = ipt
            self.country_code2 = cc2
            self.country_code3  = cc3
            self.country_name = cname

def import_csv(csvfile):
    """ import entries from csv file"""
    import csv
    reader = csv.reader(open(csvfile))
    count = 0
    session = create_session()
    for ipf, ipt, cc2, cc3, cname in reader:
        count += 1
        obj = IPToCountry(ipf, ipt, cc2, cc3,cname)
        session.save(obj)
        if count % 10000 == 0:
            print count
        print count , "inserted. :)"
    session.flush()
    del reader

ip2c_mapper = mapper(IPToCountry, iptocountry)

Imported these into controllers/blog.py where we need them. Attached them to the global c variable.

   from chiselman.models.iptocountry import iptocountry, IPToCountry
   .
   .
   class BlogController(BaseController):
       ...
       @dispatch_on(POST='create_comment')
       def blog_post(self,slug):
           ...
           c.iptocountry = iptocountry
           c.IPToCountry = IPToCountry
           ...

Import data

Then we need to actualy import the data by running import_csv, to do this get to the paster shell. Paster shell is very nice. It is in same league as django-admin shell and serves the same purpose.

Enter the following commands which calls the wanted function.

>>>import chiselman.models.iptocountry as ip2c

#create the table in the database

>>>ip2c.iptocountry.create()
#import data from the csv file.

>>>ip2c.import_csv('/home/pradeepgowda/projects/chiselman/ip-to-country.csv')
# wait for a while the import to complete(> 67,000 records.)

presenting the flags

This where Pylons differs significantly from Django. While Django best practice is to create a filter, you can do with a simple Myghty component in Pylons.

IMO, writing a component feels much more "with the flow" than writing a filter. Somehow creating a filter feels like its more work than writing a quick component. One reason could be that, you can write the code that does the 'filtering' inline with your existing code; see it working and refactor it out into a proper component later.

The code below shows the comment display component which takes care of displaying the comments.

from markdown import markdown
from sqlalchemy import and_, create_session

comment
</%args>

body = markdown(comment.message)
if not comment.web.startswith('http://'):
  url = 'http://'+comment.web
else:
  url = comment.web

session=create_session()
iptc = cc2 = ccname = None
if len(comment.ip) > 0:
  ip = m.comp('ip2long',ip=comment.ip)
  if ip:
    iptc = session.query(c.IPToCountry).selectfirst(and_(c.IPToCountry.c.ip_from = ip))

if iptc:
        cc2 = iptc.country_code2.lower()
        ccname = iptc.country_name.lower()
 </%python>

% if cc2:
  .gif" alt=" flag"/>
%





</%def>

ip2long is the function which converts the ip address from xxx.xxx.xxx.xxx format into a integer. I've copied that function verbatim from Gregory's code and made it into a myghty module.

ip
  </%args>

    ip_array = ip.split('.')
    ip_long = int(ip_array[0]) * 16777216 + int(ip_array[1]) * 65536 +  int(ip_array[2]) * 256 + int(ip_array[3])
    return ip_long
  </%python>
</%def>

You can see this feature in action here .