GPSFS Utility Program

[MIB.42]

FYI I have 512 mb on this machine so it isnt a workhorse... I can check out the memory dump if you need. I do have access to full blown racked servers if I need to test it on a workhorse.

FW

I just tried a 3GB map with 349524 tiles, it loaded it quite nicely and is currently analysing, but I see no memory footprint increase. I spent extra efforts in making sure, that there is no memory leak...

Anyways, for this 3GB map, my memory footprint was around 30MB only... Can you check how much free memory you have? My favourite tool for it is from http://www.prcview.com

Anyways, can you upload your map somewhere, so I can have a look?

[FatesWebb]

I just tried a 3GB map with 349524 tiles, it loaded it quite nicely and is currently analysing, but I see no memory footprint increase. I spent extra efforts in making sure, that there is no memory leak...

Anyways, for this 3GB map, my memory footprint was around 30MB only... Can you check how much free memory you have? My favourite tool for it is from http://www.prcview.com

Anyways, can you upload your map somewhere, so I can have a look?

OK, so I am going to copy it to a racked server with a Xeon 2.8GHZ and 4gb ram, and test it there, here is a copy of it if you want to test it yourself. (sending a private message with the location) I will take it down after you get it, because this is not a permanent place to host this file.

[FatesWebb]

ok so I have loaded it on the racked server, and it does not crash where it did on this machine. It is analyzing. I am not sure about the steps to take to use the program.

Do I have it correct that it will attempt to re-compress the images to make the map smaller? is there a good tutorial on what to do once you have loaded the map? I clicked analyze but I am not sure if there is more to it than that.

Do you want me to continue chasing why it would not run on this machine/reboot and retry or just use a beefier box. it is a 2.6GHZ running winxp and 500mb ram...

FW

[MIB.42]

ok so I have loaded it on the racked server, and it does not crash where it did on this machine. It is analyzing. I am not sure about the steps to take to use the program.

Do I have it correct that it will attempt to re-compress the images to make the map smaller? is there a good tutorial on what to do once you have loaded the map? I clicked analyze but I am not sure if there is more to it than that.

Do you want me to continue chasing why it would not run on this machine/reboot and retry or just use a beefier box. it is a 2.6GHZ running winxp and 500mb ram...

FW

Double Click the window title bar for instructions. It even says so! ;-)

I have made lots of modifications already, so it wouldn't help if we chase this with the v0.1...

Thanks.

[FatesWebb]

cool, trying it now..

what is your experience? that max compression is the way to go? or does it cause too many artifacts?

Also when it says emp, that is empty, so seemingly useless....

but on/off does off mean it removes it from the compression that the util does? or that it removes it from the archive altogether? .....(EDIT) ok wait I think I figured that one out by reading this thread, it means disable it from the processing? because I see in this thread where it says that EMPs only take up 8 bytes anyways(/EDIT)

testing with max compression to see if I like it that way...

This is fun, thanks!

FW

[FatesWebb]

Hmm, I think something went wrong, after running it a full compression here is what I was left with.... It looks like it started to work, but somehow made 4 too big, and the files I ended up with are just as big as the source files...

[MIB.42]

cool, trying it now..

what is your experience? that max compression is the way to go? or does it cause too many artifacts?

I usually go with compressions at around 40~50% of the slider ( translates to ~65% JPG )

Also when it says emp, that is empty, so seemingly useless....

It simply shows, that there is no title assosicated with a given position. Quite an important feature for analysis, but useless for recompression. GPSFS_util is not a recompression tool only, but an analysis one as well...

but on/off does off mean it removes it from the compression that the util does? or that it removes it from the archive altogether? .....(EDIT) ok wait I think I figured that one out by reading this thread, it means disable it from the processing? because I see in this thread where it says that EMPs only take up 8 bytes anyways(/EDIT)

Yes, when you turn off a tile, that tile will be omitted from the resulting conversion. This in turn will produce and EMPTY tile reference in the newly generated GPSFS files.

Also, if your original GPSFS was a PNG based file ( it will show after you load it in the information bar on the bottom left, recompression is not always good, as conversion from PNG to JPG can vary quite much...
v0.3 will have PNG recompression to PNG...

I am almost done with v0.2, I am only running tests now. It is a major improvement... I plan on releasing it within 24hours.

Cheers.

[Codaz]

When does version 3 come. I hope PNG compression is better.

Within some days i have 1 gb memory, no more 512 mb so i hope analyzing goes a lot faster.

[MIB.42]

v0.2 is attached.

v0.2 changes :
- By popular demand; Evaluated 16bit JPG compression, no benefit...
- Implemented 'FastPass' for jpg compressions.
Improvement on analysis time is ~20~30%, but reduces size prediction. Still worth it...
- Select map FOLDER instead of the GPSFS file
- Multi-tile preview window with navigations ( control buttons and mouse )
- Better logistics on task interruptions
- Play sound after analysis / convert is done
- Split the ETA computation into "empty" and "normal" tiles and compute the result accordingly
to provide more accurate ETA. This is still not 100% accurate, as it would require more system resources
which I prefer using on speeding up the conversion process instead...
- Added hour on top of the existing minutes:seconds in ETA


Again,

The following text can also be accessed by clicking on the title bar of the window.

Usage :

1. Click on [Open] and select the map directory where the GPSFS files are.
2. Enable/disable tiles, levels...
[Levels] for omitting zoom-levels.
[On/Off] to enable/disable (turn on/off) individually selected tiles.
( multi selection is available in the listbox )
3. Click on [Analyze] - May take quite some time... Enabling [No Visual] can speed it up...
4. Adjust compression settings ( after completed analysis only )
Scrollbar : choose approximate target size (compression ratio setting).
Enabling/disabling selected tiles/levels will reduce size approximations. Re-Analyze helps...
5. Click on [Create] and choose/create a folder where the new map will be stored.
- Again, might take a while, but not as long as the analysis...

Others:
'Re-Analyze' is there if you enabled/disabled several tiles ( changed lots of settings ) and already
had an analysis done before. ( This will guarantee a more precise size approximation on the files to
be created... )

Navigation :
Besides the obvious buttons in the 'Navigation' area ( I/O = Zoom In/Out, U/D/R/L = Move Up/Down/Right/Left )
there are mouse controls as well :
- Clicking with the left button on the tiles around the central tile will result in repositioning the tiles
( move to the selected direction )
- Clicking with the left button on the central tile will toggle its status (enabled/disabled).
- Clicking with the right button on the central tile will pop up a detailed information on the tile.
- Clicking with the right button on the upper tiles and tile to the right from the central one will zoom in.
- Clicking with the right button on the lower tiles and tile to the left from the central one will zoom out.

Limitations :
- Only JPG based GPSFS will be created, PNG based ones will be converted to JPGs. (v0.3 will have PNG recompression )
- Be patient, it may take a long time to do the proper analysis...
- If there are insane amount of tiles, the listbox will be disabled.
( Individual tile selection can be done with the scrollbar / navigations )
- After the analysis is done, you can elect to create several outputs by creating maps into different directories...
The analysis has to be done once only, while creation with different settings can be done multiple times...
- Completion time predictions are very rough and can be inaccurate if a map has lots of missing tiles.
( A more precise prediction would consume a lot more system resources... )

Cheers,
MIB.42

[Codaz]

Great ! Many of my additions have been added

But the 16 bit addition, why doesn't it have benefit ?
Also i like the Fastpass !!

Is PNG to JPG conversion bad for file size ? Because i think JPG is larger than PNG.

I don't like the levels 1 to 10 placed under eachother, make that like version 0.1 it saves space on your desktop !

But there's a BIG bug in 0.2

I analysed the same old map i did in 0.1 and it was there 9.44 mb. If i analyse it in 0.2 it becomes 10.21 mb !

Second BIG bug:

What is this after Analyze ?



Not displayed correctly !

Third:

If i have recursion in map dirs EG:

C:\Maps\_blabla

and

C:\Maps\_blabla\_newrecompressed

the software takes em both OR can't analyze it.