[PonyORM-list] MYSQL Unicode / Latin-1 Encoding Problem

[Jake Austwick]

Hey,

I am trying to store some data that I'm scraping with Scrapy in MySQL, usually I use Postgres however not available this time due to client requirements.

The error I'm receiving is:
File "/Library/Python/2.7/site-packages/pony/orm/core.py", line 1263, in check
   else: return converter.validate(val)
 File "/Library/Python/2.7/site-packages/pony/orm/dbapiprovider.py", line 387, in validate
   elif isinstance(val, unicode): val = val.encode(converter.encoding)
exceptions.UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2019' in position 1001: ordinal not in range(256)


However I'm unsure why it's trying to encode as latin-1, I've tried everything to try and get rid of it:

My MYSQL database and columns are set to UTF8
I have the following at the top of my python file:



import sys
reload(sys);
sys.setdefaultencoding("utf8")


I'm creating my DB connection like this:
db = Database('mysql', db="xxx", user="root", charset='utf8', use_unicode=True)



Was wondering if you had any insight on the issue?

Thanks,
-- 
Jake Austwick



-------------- next part --------------
An HTML attachment was scrubbed...
URL: </ponyorm-list/attachments/20140817/7a0a254f/attachment.html>