While building a FAERS 2016–2025 dataset and ran into what looks like a character encoding issue in the parsed drug table. After downloading/parsing FAERS data with the package, calling View(drug) in RStudio failed with:
Error in nchar(col, type = "width") :
invalid multibyte string, element 16213314
Inspecting the offending row showed that the issue appears to come from the lot_num field:
Output:
year quarter primaryid caseid drug_seq role_cod drugname prod_ai val_vbm route
<int> <char> <char> <char> <int> <char> <char> <char> <int> <char>
1: 2019 q3 165897511 16589751 1 PS KINERET ANAKINRA 1 Subcutaneous
dose_vbm cum_dose_chr cum_dose_unit dechal rechal lot_num exp_dt nda_num dose_amt
<char> <num> <char> <char> <char> <char> <char> <char> <char>
1: <NA> NA <NA> <NA> Y 33298?\xad1B <NA> 103950 100
dose_unit dose_form dose_freq
<char> <char> <char>
1: MG <NA> QD
The problematic value appears to be:
This seems to contain a malformed or non-UTF-8 byte sequence / soft hyphen-like character, which causes downstream display/string-width operations to fail.
Would it make sense for faers_parse() or the relevant table parsing helper to normalize character columns to valid UTF-8 before returning the parsed tables? This would help avoid failures in View(), nchar(type = "width"), and potentially downstream export to Parquet or other formats that expect valid UTF-8 strings.
While building a FAERS 2016–2025 dataset and ran into what looks like a character encoding issue in the parsed
drugtable. After downloading/parsing FAERS data with the package, callingView(drug)in RStudio failed with:Inspecting the offending row showed that the issue appears to come from the
lot_numfield:Output:
The problematic value appears to be:
This seems to contain a malformed or non-UTF-8 byte sequence / soft hyphen-like character, which causes downstream display/string-width operations to fail.
Would it make sense for
faers_parse()or the relevant table parsing helper to normalize character columns to valid UTF-8 before returning the parsed tables? This would help avoid failures inView(),nchar(type = "width"), and potentially downstream export to Parquet or other formats that expect valid UTF-8 strings.