invalid UTF-8 string in argument 1! (TrimStr)

Discuss any general programming issues here
Post Reply
Bugala
Posts: 1178
Joined: Sun Feb 14, 2010 7:11 pm

invalid UTF-8 string in argument 1! (TrimStr)

Post by Bugala »

I am trying to Trim a string containing a Euro-sign, and I get following error:

invalid UTF-8 string in argument 1!

How do I fix this?
plouf
Posts: 467
Joined: Sun Feb 04, 2018 11:51 pm
Location: Athens,Greece

Re: invalid UTF-8 string in argument 1! (TrimStr)

Post by plouf »

the following has no problem here

Code: Select all

a$ = TrimStr("€€€€Ελληνικά€€€€€","€",False)
DebugPrint(a$)
a$ = TrimStr("€€€€Ελληνικά€€€€€","€",True)
DebugPrint(a$)
can you post an example ?
also can you try with a command line with Notepad edited text ? (in case it is related to your IDE problem)
Christos
Bugala
Posts: 1178
Joined: Sun Feb 14, 2010 7:11 pm

Re: invalid UTF-8 string in argument 1! (TrimStr)

Post by Bugala »

I took Data from here: https://coinmarketcap.com/currencies/te ... ical-data/

In practice the problem is with the prices having Euro-signs in front of them.

I have first copy-pasted this text into Open Office Calc and saved it as CSV-file.

Then using:

Code: Select all

Function p_LoadCSVData(filename$)
OpenFile(1, filename$)
csvdata={}
While Not Eof(1) Do InsertItem(csvdata, ReadLine(1))
CloseFile(1)
Return(csvdata)
EndFunction
after which using something like:

Code: Select all

csvseparator = ";"
			DataCategories = { "Time", "Open", "High", "Low", "Close", "Volume", "Market Cap" }
			
For Local n = 1 To TableItems(csvdata)-1
	
	array, c = SplitStr(csvdata[n], csvseparator)
	Local NewItem = {}
	For k = 0 To c-1
			NewItem[DataCategories[k]] = array[k]
	Next
	InsertItem(CoinData, NewItem)
Next
and then finally getting them one by one and trying to fix them, as example getting the error when trying to remove the '"' around the numbers:

Code: Select all

For Local n = 0 To TableItems(CoinData)-1
ReplaceStr(CoinData[n]["Close"], "\"", "")
But errors reason is in EUR-sign I suppose.

Havent tried with Notepad, especially since I already figured an alternative way to get rid of EUR-sign, I was simply using:

Code: Select all

EUR = UnrightStr(EUR, 1, #ENCODING_RAW)
which removes the first character from the right, which is the EUR-sign.

However, in future there could come similar situation when trouble sign is not on the edge or there could be varying amounts of them or something, so question remains, how to handle a situation like this?

For example, although I was able to use the #ENCODING_RAW, and I suppose I could use in ReplaceStr() as well, but how am I able to put the Euro-sign into ReplaceString at all?

I mean, ReplaceStr(String, "Eur-sign", "")

How can I figure out how to make that EUR-sign in the first place if it is not some standard thing?

I suppose your Notepad system might work, as in save on notepad and then copy-paste from there too.
plouf
Posts: 467
Joined: Sun Feb 04, 2018 11:51 pm
Location: Athens,Greece

Re: invalid UTF-8 string in argument 1! (TrimStr)

Post by plouf »

Another one "trapped" in crypto investments :-)

Anyway iam not in office. But are you sure the returning data is properly utf8 ? The use of "raw" supsects

Maybe export data and compare it with a utf8 equal text to check
Christos
Bugala
Posts: 1178
Joined: Sun Feb 14, 2010 7:11 pm

Re: invalid UTF-8 string in argument 1! (TrimStr)

Post by Bugala »

I have no idea.

All I did was that I copy-pasted text to Open Office calc and saved it as csv, so no idea how it saves it.

Also, that is basically how I want it to work, since I am doing this software for my yearly book keeping, so I try to keep in such way that I need to do as little as possible, which means that already copy-pasting prices and saving them as csv is basically bit too much, and I rather not have any more extra steps on it or there are more possibilities for error when I do it the next year again and have already forgotten everything.
plouf
Posts: 467
Joined: Sun Feb 04, 2018 11:51 pm
Location: Athens,Greece

Re: invalid UTF-8 string in argument 1! (TrimStr)

Post by plouf »

In order to help ,even to andreas to fix a possible bug.
I think would help if you upload a "problematic" text exported by OOo

Btw hasnt OOo has encoding in save options ? ( To properly select UTF8

Complete out of topic. But prefer LibreOffice is by fas more upadated/ maintance
Christos
Bugala
Posts: 1178
Joined: Sun Feb 14, 2010 7:11 pm

Re: invalid UTF-8 string in argument 1! (TrimStr)

Post by Bugala »

I am actually using LibreOffice, just have got used to calling it OpenOffice despite the name change.

But you are right, just checked that Saving page and it lets you choose the encoding. I guess that will solve my problem in the future too. Thanks!

If I for some reason run into a situation where this saving format won't solve my problem, I will then make another reply to this thread maybe.
Post Reply