reading an UTF16 text file

[xabierpayet]

is possible read an UTF 16LE file in hollywood?
i´m attempting to read this file ----> http://xabierpayet.com/ext.txt
but only have garbage characters in hollywood, any idea?

[plouf]

not native
hollywood is utf8 only

must create a custom routine or use an external tool, like iconv

[Flinx]

Here is a function for reading text that I wrote without much Hollywood experience. Probably there is a faster and easier way to do this, but since it works, I haven't changed anything. However, it only reads UCS-2 because that was sufficient in my case. Of course, you could also extend it to UTF-16 with more than two bytes.

Code: Select all

Function p_ReadLine(f, UCS2LE)
	Local l$=""
	While Not Eof(f)
		If UCS2LE
			c$=Chr(ReadShort(f,#IO_LITTLEENDIAN))
		Else
			c$=Chr(ReadShort(f))
		EndIf
		If c$<>"\r" And c$<>"\n" Then l$=l$ .. c$
		If c$="\n" Then Break
	Wend
	Return(l$)
EndFunction


id=OpenFile(Nil, "ext.txt")

Local bom=ReadShort(id)
If bom=0xFEFF
	UCS2BEBOM=True
ElseIf bom=0xFFFE
	UCS2LEBOM=True
Else
	Seek(id, 0)
	If ReadByte(id)=0xEF
		If ReadByte(id)=0xBB
			If ReadByte(id)=0xBF
				UTF8BOM=True
			EndIf
		EndIf
	EndIf
	If Not UTF8BOM
		Seek(id, 0)
	EndIf
EndIf

While Not Eof(id)
	If UCS2BEBOM Or UCS2LEBOM
		CurrentLine$=p_ReadLine(id, UCS2LEBOM)
	Else
		CurrentLine$=ReadLine(id)
	EndIf
	DebugPrint(CurrentLine$)
Wend
CloseFile(id)

[xabierpayet]

ok, thanks i must attempt read it with holliwood, but really i don´t see how yet

[xabierpayet]

perfect, i check it now

[xabierpayet]

Here is a function for reading text that I wrote without much Hollywood experience. Probably there is a faster and easier way to do this, but since it works, I haven't changed anything. However, it only reads UCS-2 because that was sufficient in my case. Of course, you could also extend it to UTF-16 with more than two bytes.

Code: Select all

Function p_ReadLine(f, UCS2LE)
	Local l$=""
	While Not Eof(f)
		If UCS2LE
			c$=Chr(ReadShort(f,#IO_LITTLEENDIAN))
		Else
			c$=Chr(ReadShort(f))
		EndIf
		If c$<>"\r" And c$<>"\n" Then l$=l$ .. c$
		If c$="\n" Then Break
	Wend
	Return(l$)
EndFunction


id=OpenFile(Nil, "ext.txt")

Local bom=ReadShort(id)
If bom=0xFEFF
	UCS2BEBOM=True
ElseIf bom=0xFFFE
	UCS2LEBOM=True
Else
	Seek(id, 0)
	If ReadByte(id)=0xEF
		If ReadByte(id)=0xBB
			If ReadByte(id)=0xBF
				UTF8BOM=True
			EndIf
		EndIf
	EndIf
	If Not UTF8BOM
		Seek(id, 0)
	EndIf
EndIf

While Not Eof(id)
	If UCS2BEBOM Or UCS2LEBOM
		CurrentLine$=p_ReadLine(id, UCS2LEBOM)
	Else
		CurrentLine$=ReadLine(id)
	EndIf
	DebugPrint(CurrentLine$)
Wend
CloseFile(id)

Uauuuuuuuuuuu, work perfect, at the first attempt, thank you for the code