just wanted to say thanks!! this tool made my 2gb map half the size, (it could have even made it 600 mb but I stuck with 1gb... Thanks!!!! it looks perfectly fine... I am going to use this on all my maps!!
FW
Printable View
just wanted to say thanks!! this tool made my 2gb map half the size, (it could have even made it 600 mb but I stuck with 1gb... Thanks!!!! it looks perfectly fine... I am going to use this on all my maps!!
FW
How long did that take to process Fateswebb ?
To be honest I don't know, I put it on the server, which has 4gb of ram, and a 2.8ghz zeon processor, and just let it run... didn't time it. but I would guess about 20 minutes to load, about 2 hours to analyze, and about 1.5 hour to compress...
4gb of ram, wtf that's amazing.
Hope MIB.42 can make his software faster for normal earth pc users :) Not everybody has a server in his home with that specs :)
GPSFS_util seldom uses more than 64MB ( as in megabyte ) memory!Quote:
Originally Posted by Codaz
It is simply CPU bound due to the compression works...
I work in a data center, actually 4 data centers, so I have access to lots of hardware. I loaded it on one of the server.
In September buys my brother a Quad Core system.Quote:
Originally Posted by MIB.42
Will this speed up the compression or does it use only a single core ?
Since I only have a single core machine, I have no idea - just yet - on how to code for multi-cores... I'll have a quick look later...Quote:
Originally Posted by Codaz
Maybe you can use code from Wprime, a dualcore testing tool.Quote:
Originally Posted by MIB.42
See www.wprime.net
I googled some things, i need to know which coding language you are using. I Guessed Python but found out that it was probably C++
See these links:
http://mail.python.org/pipermail/pyt...er/405775.html
And this:
http://www.open-std.org/jtc1/sc22/wg...006/n2094.html
But remember this:
Why Doesn’t C++ Contain Built-In Support for Multithreading?
C++ does not contain any built-in support for multithreaded applications. Instead, it relies entirely upon the operating system to provide this feature. Given that both Java and C# provide built-in support for multithreading, it is natural to ask why this isn’t also the case for C++. The answers are efficiency, control, and the range of applications to which C++ is applied. Let’s examine each.
Source:
http://www.devarticles.com/c/a/Cplus...hreading-in-C/
Simple script:
http://www.thescripts.com/forum/thread631573.html
If you are coding in Visual Studio (2005) then read this:
http://msdn2.microsoft.com/en-us/lib...9t(VS.80).aspx
and this (don't mind the German):
http://www.microsoft.com/germany/msd...x?id=118757738
Hope that helps a bit, maybe not :P
MIB.42 are you checking the bikini babes on vacation or ?
We thought you left this project !
When does the .3 version pop-up, with PNG support ?
Thanks !
@MIB.42, I finally got some time to try out this program, and I must say, it's pretty cool.
I used to do similar thing with cygwin scripts and image-magic, but, of course, it was not as user friendly...
Ok, bikini babes are all accounted for. I am back on the project. I still have to think of an effective way of color reducing those png files to 16 though... My present routines either are memory hogs or CPU ones... must... think... more...Quote:
Originally Posted by Codaz
Good luck, i'm still converting my 550 mb map on a spare celeron computer i found.
Way too slow but ok.
Just status update... Getting closer to a decent - and relatively fast - color reduction... It's not trivial... Here is a snapshot on the analysis code to fine-tune the algorithm :
nice!
Looks almost as complex as photoshop.. :-)
Thanks, actually, it does a lot more than photoshop... ;-)Quote:
Originally Posted by deniska
Anyways, I thought I go after the tools, as you guys seem to have the PSP code quite well covered... ;-)
Cheers, MIB.42
I was hopping you'd do some psp coding too...
I may need help in following areas:
- psp-290 -> nmea converter
- stick to road functionality (check current pixel and see if you are on a road, if not try to look around for road colors..
- route calculator (based on google maps imagery)
anyway.. it's up to you!
great work with the tool!
Great MIB.42 !
Could you make this simple function into 0.3 ?
Comparison table of sizes. If i resize a picture in Photoshop it says:
At quality 100 the size will be ... At quality 50 the size will be ....
Now in your program could you make a comparison (with radio buttons) to choose if you want to have JPEG conversion, or PNG conversion)
So for example:
Your map with 60% compression will be 500 mb with JPEG and 400 mb with PNG. So we can choose what's best for us. Not only the file quality but also file size comparison :)
This is calculated on the fly after conversion.
Is there any up to date GPSFS documentation available ? I have read mapthis parsing code, but it is a bit ennoying... So if you have just some clues they are welcome :)
I would like to understand the "basezoom", is it to tell the zoom used for the map first level of tiles (ie the first 4 tiles if I understood properly) ?
Regarding the tile index in the early documentation I found it is said it is an offset, but as there are several files I guess there must be a file index as well.
Regarding the tiles size is it constraint ? (ie 256x256) or is it free ?
Thanks for your help ;)
Finally I found the missing informations in gmdl source code ;) (GPSFS.cs FYI)Quote:
Originally Posted by Ixar
What I was missing is the 2 formats for GPSFS, I was only aware of the 1st one. Now things are more clear.
Best Regards,
Ixar
Some documentation is available here:
http://en.wikibooks.org/wiki/Map_This!
in case you still have questions..
Thanks I have already read the GPSFS part in that wikibook.Quote:
Originally Posted by deniska
I still have one question regarding the GPSFS file split: what is the motivation ? Is there any rule behind that split or is it arbitrary ?
Ixar, as I recall seeking within a single file was a bit slow - so that's the reason for the 5 file setup (this is hard coded, so a 4 or 6 file setup will not work). The split is on size to try to get the files to be of comparable size (the last file ends up a bit larger due to rounding/left over tiles, but since the discrepancy is minor there hasn't really been a reason to fix that).
Also, keep in mind Int64 vs Int32s being used in different fields.
Thanks for the explanation ;)Quote:
Originally Posted by in7ane
I think I have now understood the format, with the IDXFile and IDXTile on 32 bit. One thing that is not specified is the endianess, but I guess it is little endian as is the PSP.
I will soon try my own maps.
Ideally I would like to implement a new feature in MapThis to be able to merge maps "on the fly". This way if a tile is not available in a map, it take the one in another map.
But let's see first if I managed to make something with GPSFS :P
Any progress MIB.42 or is it too difficult to make ?
Oh man, I'm so swamp'd at work and school also started for the kiddos... I can write like 10 lines a code a day...Quote:
Originally Posted by Codaz
Nevertheless, I am getting there. The HSV based base-color reduction seems to work ( I can already reduce to 32 with minor bugs ) and I will only need to write the final reduction code. There is still some "tweaking" involved in choosing the human perception baselines, but it's getting really good. I stop'd making predictions on when this will be ready, but I am at the moment now that I can see the light at the end of the tunnel... I was actually thinking of taking a few days of vacation ( from work ) to finish this... *sigh*
Lol, is it CPU intensive like u said ?
Good luck with the kiddos and work.
Just a quick update, the PNG color reduction finally works now and it's pretty darn fast and only uses about 1/2 MB..., I have started integrating it into the Utility Program... I'll keep you posted...Quote:
Originally Posted by MIB.42
Sorry for the delay, libpng keeps crashing after a thousand or so writes... I've checked non-png version, it works, and isolated the problem into a separate app and I am now debugging the png ( actually the zlib )... I love how things we take for granted don't work... :(Quote:
Originally Posted by MIB.42
How long do you need to fix it do you think ?
I've isolated and separated the png read part and decoupled from write. It still is crashing in zlib... After I manage to fix it, I am rewriting the PNG handler, so that there is no "Analysis" phase, as there is no need for it... When a map is PNG based, it will "simply" generate a 16 color reduced map... I really want to finish this as I'd love to work on something else as well, so the motivation is really high on finishing it asap....Quote:
Originally Posted by Codaz
I'll keep you posted...
All right, I finally had 4 hours of uninterrupted coding time... Found the bug ( needless to say, it was in my code, but it was so elusive, that it crashed zlib and by debugging libpng which led to debugging zlib gave me the hints for fixing it... *sigh* )
Anyways, if you open a map which is PNG based, it will offer a "Q-Create" button. This bypasses the analyze part and starts creating the 16 color reduced map. Since this uses only a guesstimate on the final size, GPSFS4 might be a bit out of sync with the rest, but it should still work...
If you want really nice evenly spaced GPSFS files, run the "Analyze" before the "Create".
zlib and libpng is not needed as they are linked in statically...
The 16 color PNGs reduce the maps to 45%~65% of the original sizes...
Cheers,
MIB.42
PS: I haven't tested it extensively, so let me know if you find "bugs"... Also, there is a minor memory leak ( ~20KB/1000 tiles) I will track down as soon as I have time...
PS2: I have found a minor bug in the size computations, thus the rev1 attachment update
cool!
I'll give it a try and let you know!
I tried it on a few, relatively small maps and I must say - I am impressed!
IMHO, the quality is much better than via ->jpeg conversion. The compression artifacts are hardly visible on the most detailed zoom levels..
Good job!
hi MIB.42
it's working great for me
but there is 1 thing i'm on a old monitor 800/600
and when i use it it doesn't fit to the screen
can u make it fitting the screen by pressing the square button???
thnx for the progr.johoo
Glad it works. Any "burning" issues I can help with?Quote:
Originally Posted by deniska
That's a bit tough, since 3 tiles are already 768 pixels and there is no menu or anything just yet... Hmmm, I may put together a single-tile version...Quote:
Originally Posted by johoo
I may need some help with Vector maps implementation.. and gu routines..Quote:
Originally Posted by MIB.42
I'll post the latest code snapshot once I clean it up a bit...
basically, I already re-wrote most of that stuff,but it has some minor issues...
Beautiful that coders ask eachothers help.
MIB i will try the program out in a few seconds !
I'll report ASAP.
What's on stake for version 0.4 ?
========
Report:
Program is working VERY well, BUT i can't set the image quality of PNG maps with a slider or something.
On JPEG maps i can set the end size with the slider.
Also the Qcreate doesn't give an estimate of the size it will be.
A bit confusing is it now. It would be better if there was a build in comparison chart that says:
Process the map now and you get:
JPEG normal = 50 mb.
JPEG max = 20 mb
PNG normal (current) = 50 mb.
PNG max (16bit) = 18 mb.
Or something like that.
=====
I analyzed a 54,1 mb PNG map which gives with no compression a 82 mb JPEG map. WOW ! How does that come ?
If i say compress it to jpeg max compression it will look much worse (64 mb) than png 16 bit color which is only 28,1 mb.
So great job done, map is from 54 mb to 28. With png > jpeg on max it will only go from 54 to 64 and it looks blocky.
Why is the estimate size on jpeg conversion not right ? It says with max jpeg compression (slider set) it will be 46 mb. But i viewed the folder and it was 64,6 !
I tried some things out with photoshop on several tiles and other images. Wouldn't it be good to do a 8 color GIF addon because it will give good viewable results.
8 bit diffused with restriction (216 web colors).