[PonyORM-list] MYSQL Unicode / Latin-1 Encoding Problem

Jake Austwick jake at serpiq.com
Sun Aug 17 14:38:56 UTC 2014


I am trying to store some data that I'm scraping with Scrapy in MySQL, usually I use Postgres however not available this time due to client requirements.

The error I'm receiving is:
File "/Library/Python/2.7/site-packages/pony/orm/core.py", line 1263, in check
   else: return converter.validate(val)
 File "/Library/Python/2.7/site-packages/pony/orm/dbapiprovider.py", line 387, in validate
   elif isinstance(val, unicode): val = val.encode(converter.encoding)
exceptions.UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2019' in position 1001: ordinal not in range(256)

However I'm unsure why it's trying to encode as latin-1, I've tried everything to try and get rid of it:

My MYSQL database and columns are set to UTF8
I have the following at the top of my python file:

import sys

I'm creating my DB connection like this:
db = Database('mysql', db="xxx", user="root", charset='utf8', use_unicode=True)

Was wondering if you had any insight on the issue?

Jake Austwick

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </ponyorm-list/attachments/20140817/7a0a254f/attachment.html>

More information about the ponyorm-list mailing list