Page 1 of 1

PrimalScript 2019 problem with unicode character

Posted: Mon Dec 17, 2018 1:45 pm
by gmichaud
Product: PrimalScript (64 Bit)
Build: v7.4.120
OS: Windows 10 Pro (64 Bit)
Build: v10.0.16299

At least one Unicode character that we have used in our scripts for many years causes the 2019 version to stop reading the script file, sometimes with gibberish after the character. The 2018 and 2014 versions read everything just fine.

The right double arrow is particularly troublesome:
" ยป "
In a comment it seems to simply stop reading the script. In code it causes the odd error you will find in the attached file, as well as an error that some lines are over 8128 characters long (they are NOT).
UnicodeError.zip
(41.1 KiB) Downloaded 204 times
Thanks,

Gerald Michaud
gmichaud@dxc.com

BTW, the dropdown on this web page to "select code" type doesn't seem to have a selection for vbscript. We make a LOT of vbscripts. Hopefully some day we'll move to powershell, but for the foreseeable future it's all vbscript.

Re: PrimalScript 2019 problem with unicode character

Posted: Mon Dec 17, 2018 2:43 pm
by Olga_B
Thanks for reporting the issue.
The issue has been sent to the proper place for evaluation and correction. When a fix is issued with a service build, it will be listed in the change log.

Re: PrimalScript 2019 problem with unicode character

Posted: Mon Dec 17, 2018 8:53 pm
by Alexander Riedel
It's fixed and in QA. There is new code to detect BOMs (Byte Order Marks) mid file because some folks started to concatenate unicode files with the good old DOS copy command. That created problems as you might imagine. The detection code was flawed though, so that caused your problem. Thought maybe you are curious :D

Re: PrimalScript 2019 problem with unicode character

Posted: Tue Dec 18, 2018 12:51 pm
by gmichaud
Thanks for the quick resolution and extra info. I DO like to hear these details.

One thing I learned a long time ago is that no matter how hard you try to think of all the usage scenarios and all possible ways users can break something, they'll still surprise you!

Any idea how long this will remain in QA? I'd like to provide an ETA to the rest or our team.

Thanks,
Gerry

Re: PrimalScript 2019 problem with unicode character

Posted: Tue Dec 18, 2018 8:01 pm
by Alexander Riedel
Not sure at the moment. There are few other changes in that build that still require some work. I will know more after today.

Re: PrimalScript 2019 problem with unicode character

Posted: Wed Dec 19, 2018 10:33 pm
by Alexander Riedel
The fix has been released in the latest service build (7.4.121).

Re: PrimalScript 2019 problem with unicode character

Posted: Thu Dec 20, 2018 6:27 am
by gmichaud
thanks. i'll test it this morning.