ReadLine() »

Discuss any general programming issues here
Post Reply
Flinx
Posts: 188
Joined: Sun Feb 14, 2021 9:54 am
Location: Germany

ReadLine() »

Post by Flinx »

Hello,
I noticed that text lines starting with the » character cause the line not to be read in correctly.
The ReadLine function generally omits the second byte if it is a 0xBB if the line has only two bytes. With more bytes strange things happen.
I suspect this has to do with searching for the BOM, although it not only occurs with the first line.

Code: Select all

Function ReadLineTest()
	OpenFile(1, "ReadLineTest.txt")
	rdline$=ReadLine(1)
	DebugPrint(rdline$)
	hx$=""
	For Local i=0 To StrLen(rdline$, #ENCODING_RAW)-1
		hx$=hx$ .. MidStr((HexStr(ByteAsc(rdline$, i))),1).." "
	Next
	DebugPrint(hx$)
EndFunction

StringToFile("  ABC 1 » ä-ö-ü-ß-Ä-Ö-Ü","ReadLineTest.txt")
ReadLineTest()
StringToFile("»ABC 2 » ä-ö-ü-ß-Ä-Ö-Ü","ReadLineTest.txt")
ReadLineTest()
StringToFile("»","ReadLineTest.txt")
ReadLineTest()
/*
For k=0x0000 To 0xffff
	StringToFile(ByteStrStr(k,#SHORT), "ReadLineTest.txt")
	ReadLineTest()
Next
*/
Ralf
User avatar
airsoftsoftwair
Posts: 5433
Joined: Fri Feb 12, 2010 2:33 pm
Location: Germany
Contact:

Re: ReadLine() »

Post by airsoftsoftwair »

Yup, clearly a bug but fixed now. As a workaround, just open the file in ISO 8859-1 encoding by passing the "Encoding" tag to OpenFile(). Then the problem shouldn't occur.

Code: Select all

- Fix: When opening a file in UTF-8 mode, ReadLine() could get confused when reading a byte that is part
  of the UTF-8 BOM ($ef, $bb, $bf) 
Post Reply