Unicode tests fail with Python 2
Created by: qauge
I have trouble getting tests right from the repository freshly cloned from master.
Some "unicode tests" won't pass with Python 2 (UnicodeDecodeError). Those are test_ndjson for csvjson, test_unicode for csvlook, test_sort_date for csvsort, and test_sort_string_reverse for csvsort.
Sample output:
ERROR: test_ndjson (tests.test_utilities.test_csvjson.TestCSVJSON)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/quentin/work/csvkit/tests/test_utilities/test_csvjson.py", line 108, in test_ndjson
'{"text": "Unicode! Σ", "float": null, "datetime": null, "boolean": null, "time": null, "date": null, "integer": null}',
File "/home/quentin/work/csvkit/tests/utils.py", line 56, in assertLines
lines = self.get_output_as_list(args)
File "/home/quentin/work/csvkit/tests/utils.py", line 44, in get_output_as_list
return self.get_output(args).split('\n')
File "/home/quentin/work/csvkit/tests/utils.py", line 37, in get_output
utility.main()
File "/home/quentin/work/csvkit/csvkit/utilities/csvjson.py", line 168, in main
table = agate.Table.from_csv(self.input_file, sniff_limit=self.args.sniff_limit, column_types=self.get_column_types(), **self.reader_kwargs)
File "/home/quentin/work/csvkit/.tox/pypy/site-packages/agate/table/__init__.py", line 362, in from_csv
contents = contents.encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position 295: ordinal not in range(128)
Is it something on my side only?