sniffer slowness
Created by: thatmattbone
I have a somewhat large csv (~35k lines) that takes quite a while to run csvstat over. It seems like the cause of the slowness is the call out to python's csv.Sniffer().sniff() which runs over the entire contents of the file.
My simple fix was to limit the sniff to the first 4096 bytes of the file. Maybe this should be a command line flag? Or maybe sniffer.py should not use csv.Sniffer().sniff() to determine the dialect if the quotechar, delimiter, etc is specified?
Either way I'm happy to make a patch.