Asynchronous Downloads

Show off your project created with Hollywood
Post Reply
djrikki
Posts: 682
Joined: Wed Apr 06, 2011 12:26 am

Asynchronous Downloads

Post by djrikki »

Hi,

What follows below is code that you can use for your projects. This particular example downloads 3 files from OS4Depot asychronously/simultaneously saving each to RAM:.

There is no error correction in place if the user should lose a connection nor am I able at this point determine the size of the remote file. As I am coding to adapt this code into the Jack App-Store I already know the size of the remote file, so you will need to find out the size of the remote file yourself- assuming its possible on the host server. Haven't looked into this possibility whatsoever so can't help at this stage however I would be interested to know how it can be achieved.

Enjoy, feel free to give me a credit in your project.

Code: Select all

/*
************************************************************
**
** Created by: CodeBench 0.26 (21.04.2012)
**
** Project: OpenConnection
**
** File: main.hws
**
** Date: 09-03-2012 20:14:47
**
************************************************************
*/

; This asychronous download example code is adapted to OS4Depot where HTTP
; redirection is in place and must be adapted if the host site doesn't use
; HTTP redirection.

Global base, data$, count, done, host$

host$ = "www.os4depot.net"

base = {}

; Userdata, FilePort, Remote Archive, Bytes Received, Total Filesize
DimStr connections[30][2][1][1][1]

For Local c = 0 to 29
    connections[c][0] = ""
    connections[c][1] = c
    connections[c][2] = ""
    connections[c][3] = 0
    connections[c][4] = 0
Next

Function base:AllocateFindFilePort(userdata)
    
    Local c, available, found
    
    If userdata = nil ; Allocate File Port
	    For c = 0 to 29
	       	If GetType(connections[c][0]) <> #LIGHTUSERDATA
	       	    available = True
	       	    Break
	       	EndIf
	    Next
	    
	    ; If no connections available return Nil, maxconnections set in DimStr connections[]
	    If available = False
	        c = Nil
	    EndIf
	
	Else ; Find File Port via userdata

		For c = 0 to 29
	       	If GetType(connections[c][0]) = #LIGHTUSERDATA
	       	    If connections[c][0] = userdata
		       	    found = True
		       	    Break
		       	EndIf
	       	EndIf
		Next
		
		If found = False
		    c = Nil
		EndIf
    EndIf
        
    Return (c)
EndFunction

Function base:DownloadFile(server$, url$, downloadpath$, filename$, filesize)
    
    Local id = base:AllocateFindFilePort()
    
    Local data$, count, done
              
	connections[id][0] = OpenConnection(nil, server$, 80)

	SendData(connections[id][0], "GET " .. url$ .. " HTTP/1.0\r\n\r\n")
	
	; This line here will not work if the site doesn't use redirection, you will need to
	; adapt this accordingly.
	data$, count, done = ReceiveData(connections[id][0], #RECEIVEALL)
	
	; Redirect
	If FindStr(data$,"302 Found") > -1
	    CloseConnection(connections[id][0])
	    redirection$ = MidStr(data$,FindStr(data$,"Location: "),StrLen(data$))
	    redirection$ = MidStr(redirection$,10,FindStr(redirection$,"\n")-11)
	    
	    server$ = ReplaceStr(redirection$, "http://", "")
	    server$ = MidStr(server$,0,FindStr(server$,"/"))
	    
	    connections[id][0] = OpenConnection(nil, server$, 80)
	    
	    SendData(connections[id][0], "GET " .. redirection$ .. " HTTP/1.0\r\n\r\n")
	    
		data$ = nil
	 
		; Skip HTTP Headers
	    While data$ = nil Or FindStr(data$,"Content-Type:") = -1
	    	data$, count, done = ReceiveData(connections[id][0], #RECEIVELINE)
	   	Wend
   	
	   	data$, count, done = ReceiveData(connections[id][0], #RECEIVELINE) ; Skip blank line

	; Adapt this for sites that don't use HTTP rdirection   		    	    
	Else   
)
	EndIf

	; Not Found
	If FindStr(data$,"404 Found") > -1
	    Return(False)
	    
	; Found - Download it
	Else
    
		connections[id][1] = OpenFile(nil,downloadpath$ .. filename$, #MODE_READWRITE)
		connections[id][2] = filename$
		connections[id][4] = filesize
	
	    Return(True)
	EndIf
EndFunction
    
Function base:BuildFile(userdata)   
	Local segment$
	segment$, count, done = ReceiveData(userdata.id, #RECEIVEBYTES, 65536) ; Read file data
	
	Local port = base:AllocateFindFilePort(userdata.id)
		
   	WriteString(connections[port][1],segment$)
			
	connections[port][3] = connections[port][3] + StrLen(segment$)
								
	If connections[port][3] = connections[port][4] Then SaveToDisk(userdata.id,port)
EndFunction
    
Function base:Keyboard(msg)
	Switch msg.action
	    Case "OnKeyDown"
	    	If msg.key = "ESC"
	    	    End()
	    	EndIf
	EndSwitch
EndFunction
        
Function base_BuildFile(msg) base:BuildFile(msg) EndFunction
Function base_KeyboarD(msg) base:Keyboard(msg) EndFunction
    
InstallEventHandler({OnReceiveData = base_BuildFile, OnKeyDown = base_Keyboard})

debugprint("Asynchronous file download demo by Richard Lake (richard@lakemarketingandevents.co.uk)")
debugprint("\nWorks with sites like OS4Depot where HTTP Redirect is in place.")
debugprint("\nOne limitation in this demo is that you must know beforehand the filesize of each download in advance.")
debugprint("\nDownloading 3 files from OS4Depot asynchronously...")
    
success = base:DownloadFile(host$,"http://" .. host$ .. "/share/game/board/africa.lha","RAM:","africa.lha",297517)
success = base:DownloadFile(host$,"http://" .. host$ .. "/share/utility/misc/ipqalc.lha","RAM:","ipqalc.lha",114081)
success = base:DownloadFile(host$,"http://" .. host$ .. "/share/emulation/gamesystem/amiarcadia.lha","RAM:","amiarcadia.lha",3454033)

Function SaveToDisk(userdata,port)
   	CloseConnection(userdata)
	CloseFile(connections[port][1])
	
	debugprint(connections[port][2] .. " saved to RAM:")
	
    connections[port][0] = ""
    connections[port][2] = ""
    connections[port][3] = 0
    connections[port][4] = 0
Endfunction
    
Repeat
    WaitEvent()
Forever
Evolve - Rapid GUI Development tool for MUI Royale and RapaGUI
http://myevolve.wordpress.com
g0blin
Posts: 77
Joined: Tue Oct 04, 2011 9:06 am

Re: Asynchronous Downloads

Post by g0blin »

NIce piece of code.

I'll look deeper into it asap and let you know if I can help.

Regards
g0blin
Post Reply