Multiple copies of linked plugins in the temporary directory

Report any Hollywood bugs here
User avatar
jPV
Posts: 600
Joined: Sat Mar 26, 2016 10:44 am
Location: RNO
Contact:

Multiple copies of linked plugins in the temporary directory

Post by jPV »

I just noticed, that Hollywood will unpack the linked plugins to SYS:Storage directory (on MorphOS at least). The problem is that I get a new copy of the plugin every time I run the program on freshly booted system. It doesn't seem to notice the previously created file, and doesn't clean the old ones either.

I have now over 20 different directories/plugins in my SYS:Storage/Hollywood directory from few days period I've been testing it. HD will run out of space this pace quite quickly...

SYS:Storage/Hollywood looks like this, and I've only been using one plugin (muiroyale) with one program:
hollywood.000000
hollywood.000001
hollywood.000002
hollywood.000003
hollywood.000004
hollywood.000005
etc...

Each directory contains the same muiroyoale.hwp file.

Another problem I noticed with this is that when I tried to run my program when booted from MorphOS CD, it failed because the boot media is (of course) write protected. Maybe it should fallback to T: in that kind of situation?

Or why not having plugins in T: in the first place in any case? I think we have enough ram for that in current HW situation? Or maybe user defineable depending how many plugins the program will be using... with just one or two plugins it sounds useless to write to HD...
User avatar
airsoftsoftwair
Posts: 5425
Joined: Fri Feb 12, 2010 2:33 pm
Location: Germany
Contact:

Re: Multiple copies of linked plugins in the temporary direc

Post by airsoftsoftwair »

Hmm, can't reproduce this here. I'm wondering what's going on. Hollywood definitely deletes the files and the directory when it quits. I've just checked this. Two questions come to mind:

1) What file system does the partition that hosts your SYS:Storage use? Is there any software that prevents other software from deleting files from SYS:?
2) Can you post a SnoopDOS log? Start logging right before you close your app. You should definitely see Hollywod trying to delete those files. But on your system this seems to fail for some odd reason.
Or why not having plugins in T: in the first place in any case?
Well, don't forget that we only have 128 MB of RAM on Efika systems. Also, not everybody has lots of RAM on OS3 and plugins like avcodec.hwp, svgimage.hwp or sqlite3.hwp can be quite large.
User avatar
jPV
Posts: 600
Joined: Sat Mar 26, 2016 10:44 am
Location: RNO
Contact:

Re: Multiple copies of linked plugins in the temporary direc

Post by jPV »

Filesystem is SFS, but maybe the problem is that I don't quit the Hollywood application separately, it's just left running when I shutdown or reboot (making a desktop enhancer in this particular case).

And in general I think it can't be assumed that Hollywood apps will always be quitted, maybe user won't bother, maybe system crashes, etc. Users definitely won't always close all applications when they need to reboot and we don't have any automatic closing of them like on some other systems.

Doesn't Hollywood check the existing files in the temp dir in any way when launched to avoid duplicates? Maybe it should clean it at that point, or I think it would be better to check found plugin versions before extracting any new ones there. If they'd be in T: they'd be automatically cleaned, but yea, I understand the points not to have them there, but then we should have some logic to clean the permanent temp dir.
User avatar
airsoftsoftwair
Posts: 5425
Joined: Fri Feb 12, 2010 2:33 pm
Location: Germany
Contact:

Re: Multiple copies of linked plugins in the temporary direc

Post by airsoftsoftwair »

jPV wrote:Filesystem is SFS, but maybe the problem is that I don't quit the Hollywood application separately, it's just left running when I shutdown or reboot (making a desktop enhancer in this particular case).
Ok, then the behaviour you are seeing makes sense of course... if Hollywood doesn't quit, there's no chance to clean up.
Doesn't Hollywood check the existing files in the temp dir in any way when launched to avoid duplicates? Maybe it should clean it at that point, or I think it would be better to check found plugin versions before extracting any new ones there. If they'd be in T: they'd be automatically cleaned, but yea, I understand the points not to have them there, but then we should have some logic to clean the permanent temp dir.
Well, this is more difficult than it might sound. Just cleaning those files isn't possible because there might be other Hollywood apps running at the same time which need those files.

Checking for duplicates sounds like a better approach but of course if you never quit your apps properly, you'll also be left with some files in SYS:Storage/Hollywood, although you'll never end up with the same file twice of course in case Hollywood checks for duplicates.
User avatar
jPV
Posts: 600
Joined: Sat Mar 26, 2016 10:44 am
Location: RNO
Contact:

Re: Multiple copies of linked plugins in the temporary direc

Post by jPV »

airsoftsoftwair wrote:Checking for duplicates sounds like a better approach but of course if you never quit your apps properly, you'll also be left with some files in SYS:Storage/Hollywood, although you'll never end up with the same file twice of course in case Hollywood checks for duplicates.
Yeah, that wouldn't be a problem if there's just one copy per plugin, or even per plugin version, but it just isn't good if files are multiplying infinite times. Can't plugins just be in fixed paths there to make checking even easier?

Current situation makes it quite impossible for me to have linked plugins as I originally planned. Having separate plugin files isn't as elegant either... it's annoying if user would like to drag the exe to WBStartup drawer or something like that. Plugin should follow or installed separately somehow... have to think better how to release the program...
User avatar
airsoftsoftwair
Posts: 5425
Joined: Fri Feb 12, 2010 2:33 pm
Location: Germany
Contact:

Re: Multiple copies of linked plugins in the temporary direc

Post by airsoftsoftwair »

Well, thinking about it further even a check for duplicates somewhat contradicts the idea of deleting those temporary plugins when the program quits. Imagine program A starts and writes plugin to the temp store. Then program B is started and detects the plugin is already there and loads it from there. Then program A is quit and deletes the plugin but program B still needs it... I guess the easiest thing is that your installer just copies the plugin to LIBS:Hollywood...
User avatar
jPV
Posts: 600
Joined: Sat Mar 26, 2016 10:44 am
Location: RNO
Contact:

Re: Multiple copies of linked plugins in the temporary direc

Post by jPV »

There wouldn't be any way to load the plugin from the executable without needing to create any temporary files?

And some other thing came in my mind... Hollywood now looks for plugins only from the progdir or libs, but both have some problems. Plugins in libs will be loaded (and memory wasted) for all Hollywood apps then, even when they don't need the plugins. And having the plugins in the progdir makes it harder to make multiplatform releases, because you can't have exes for different platforms in the same dir, because they would look the same plugin file then. But how about a new table parameter for @REQUIRE and LoadPlugin(), where you could define a new file or path to look for the plugin? For example, @REQUIRE "muiroyale", {File = "plugins/muiroyale.mos"}, or @REQUIRE "muiroyale", {Path = "plugins/morphos/"}, or something like that?
User avatar
airsoftsoftwair
Posts: 5425
Joined: Fri Feb 12, 2010 2:33 pm
Location: Germany
Contact:

Re: Multiple copies of linked plugins in the temporary direc

Post by airsoftsoftwair »

jPV wrote:There wouldn't be any way to load the plugin from the executable without needing to create any temporary files?
I think there is an InternalLoadSeg() function in dos.library which could do that but that would only work with non-compressed applets, so once again not a thorough solution.
And some other thing came in my mind... Hollywood now looks for plugins only from the progdir or libs, but both have some problems. Plugins in libs will be loaded (and memory wasted) for all Hollywood apps then, even when they don't need the plugins.
You can change this behaviour by setting the SKIPPLUGINS tooltype. If you set the following tooltype

Code: Select all

SKIPPLUGINS=*
no plugins will be loaded by default. This behaviour can also be imitated using a *.ini file on non-Amiga platforms.
And having the plugins in the progdir makes it harder to make multiplatform releases, because you can't have exes for different platforms in the same dir, because they would look the same plugin file then.
True, but you can just use subdirectories for the different platforms.
But how about a new table parameter for @REQUIRE and LoadPlugin(), where you could define a new file or path to look for the plugin? For example, @REQUIRE "muiroyale", {File = "plugins/muiroyale.mos"}, or @REQUIRE "muiroyale", {Path = "plugins/morphos/"}, or something like that?
Hmm, maybe, but there are certainly more important things to implement :)
User avatar
jPV
Posts: 600
Joined: Sat Mar 26, 2016 10:44 am
Location: RNO
Contact:

Re: Multiple copies of linked plugins in the temporary direc

Post by jPV »

airsoftsoftwair wrote:
jPV wrote:And some other thing came in my mind... Hollywood now looks for plugins only from the progdir or libs, but both have some problems. Plugins in libs will be loaded (and memory wasted) for all Hollywood apps then, even when they don't need the plugins.
You can change this behaviour by setting the SKIPPLUGINS tooltype. If you set the following tooltype

Code: Select all

SKIPPLUGINS=*
no plugins will be loaded by default. This behaviour can also be imitated using a *.ini file on non-Amiga platforms.
But I can't be modifying all other Hollywood programs' tooltypes. If user has other HW programs, and I copy plugin to Libs with my program, they all start loading it.

I start to think that it would be the simpliest solution if Hollywood would just create single copies of linked plugins to the temp dir if they don't exist, or replace older versions, and just leave them there without trying to delete them after program has been quitted. No custom per session dirs and multiple copies, just single copy which all programs would use. The difference with this and Libs would be that this wouldn't be loaded automatically unless programs have the same file linked. Plugins won't take that much space that they couldn't be leaved there, and in any case it doesn't have risk of multiplying copies which would take much more space.
User avatar
airsoftsoftwair
Posts: 5425
Joined: Fri Feb 12, 2010 2:33 pm
Location: Germany
Contact:

Re: Multiple copies of linked plugins in the temporary direc

Post by airsoftsoftwair »

jPV wrote: But I can't be modifying all other Hollywood programs' tooltypes. If user has other HW programs, and I copy plugin to Libs with my program, they all start loading it.
Actually, you can use the SKIPPLUGINS tooltype for every program compiled by Hollywood...
I start to think that it would be the simpliest solution if Hollywood would just create single copies of linked plugins to the temp dir if they don't exist, or replace older versions, and just leave them there without trying to delete them after program has been quitted. No custom per session dirs and multiple copies, just single copy which all programs would use. The difference with this and Libs would be that this wouldn't be loaded automatically unless programs have the same file linked. Plugins won't take that much space that they couldn't be leaved there, and in any case it doesn't have risk of multiplying copies which would take much more space.
Leaving temporary files in store isn't good citizen behaviour IMHO. Why are you so keen on linking plugins? The whole plugin linking feature was actually only implemented to disguise the fact that the Hollywood APK Compiler was made using RapaGUI when RapaGUI was still top secret :) Otherwise I wouldn't have implemented a plugin linker at all...

Think of plugins as shared libraries (although they aren't really shared). Nobody would want to link Amiga libraries into executables either... just install them globally in LIBS: and you're done :)
Post Reply