1.0.0 can no longer parse my utf-8 files
Created by: perbergland
After upgrading from 0.9.1 to 1.0.0, my files can no longer be parsed with csvkit.
The error message is:
Your file is not "utf-8" encoded. Please specify the correct encoding with the -e flag. Use the -v flag to see the complete error.
Attaching a sample file - mdfile.md.zip
Commands I ran on the file:
$ cat mdfile.md
login|role|email|note
---|---|---|---
handle1|member|First Last <first.last@company.com>|My note
handle2|member|First Last <first.last@company.com>|My note with ööö utf-8
handle3|member|First Läst <first.last@company.com>|My note
$ file -I mdfile.md
mdfile.md: text/plain; charset=utf-8
$ csvjson -d '|' <mdfile.md
Your file is not "utf-8" encoded. Please specify the correct encoding with the -e flag. Use the -v flag to see the complete error.
[{"login": "---", "role": "---", "email": "---", "note": "---"}, {"login": "handle1", "role": "member", "email": "First Last <first.last@company.com>", "note": "My note"}, {"login": "handle2", "role": "member", "email": "First Last <first.last@company.com>", "note":
environment:
Mac OS X 10.12.2 Python 2.7.12 pip 9.0.1
$set | grep LC_
local LC_CTYPE=C;