Go Back   Foundation 3D Forums > Techniques, Instruction > 3D
FAQ Members List Calendar Mark Forums Read

3D Tutorials and your questions on your particular 3D application. Or post questions for general 3D help.

Reply
 
Thread Tools Display Modes
Old 29th Jun 2017, 01:00 PM   #1
Starbase1
Registered Member
 
Starbase1's Avatar
 Gallery
Blender LightWave Modo Poser Terragen Vue Gimp Paint Shop Pro Photoshop
 
Join Date: Dec 2007
Location: London
Posts: 12,688
Default General/Other - Handling giant TIF images

I'm hoping someone with deeper knowldge than me of handling images will be able to answer this one...


I was very excited to spot a new very high resolution digital elevation map of the moon!
https://astrogeology.usgs.gov/search..._60N60S_512ppd


I keep hoping to get something good enough to get really close to the craters, and this just might be it.The problem is that this is a 16 bit TIF file 22 Gigabytes in size! And I can't find anything that will open it!



I tried photoshop elements, (not a chance), Affinity, (doesn't work), but though I had found a solution with the beta version of GIMP. But when it was still trying to load the file an hour after I clicked on "open" I gave up. (I have plenty of memory)


My immediate need is to crop a (much) smaller chunk of it - I suspect that once I do that it will be relatively straightforward to work with, and I can get it into a more managable format.


So, anyone got any ideas on how to cut this monster down to size?
__________________
- - - - - -
My web site: http://nick-stevens.com/
Me on twitter, @runnymonkey, https://twitter.com/runnymonkey

Starbase1 is offline   Reply With Quote
Old 29th Jun 2017, 02:51 PM   #2
Crook
Contributing Member
 
Crook's Avatar
 Gallery
LightWave Terragen ZBrush Photoshop After Effects
 
Join Date: Dec 2007
Location: Oxford, UK
Posts: 2,530
Default

https://www.imnc.in2p3.fr/pagesperso...argetifftools/
__________________
Facebook - Andy Crook
Crook is offline   Reply With Quote
Old 29th Jun 2017, 03:22 PM   #3
Starbase1
Registered Member
 
Starbase1's Avatar
 Gallery
Blender LightWave Modo Poser Terragen Vue Gimp Paint Shop Pro Photoshop
 
Join Date: Dec 2007
Location: London
Posts: 12,688
Default

My hero!

I owe you a generous portion of the beverage of your choice!
__________________
- - - - - -
My web site: http://nick-stevens.com/
Me on twitter, @runnymonkey, https://twitter.com/runnymonkey

Starbase1 is offline   Reply With Quote
Old 29th Jun 2017, 03:50 PM   #4
Starbase1
Registered Member
 
Starbase1's Avatar
 Gallery
Blender LightWave Modo Poser Terragen Vue Gimp Paint Shop Pro Photoshop
 
Join Date: Dec 2007
Location: London
Posts: 12,688
Default

Dammit, doesn't work on that file!
__________________
- - - - - -
My web site: http://nick-stevens.com/
Me on twitter, @runnymonkey, https://twitter.com/runnymonkey

Starbase1 is offline   Reply With Quote
Old 29th Jun 2017, 08:25 PM   #5
Meurig
Moderator
 
Meurig's Avatar
 Gallery
LightWave Maya Mudbox ZBrush Paint Shop Pro Photoshop Boujou Digital Fusion Shake Premiere
 
Join Date: Nov 2007
Location: Vancouver, BC
Posts: 4,856
Default

Nuke.
Meurig is offline   Reply With Quote
Old 30th Jun 2017, 08:13 AM   #6
Crook
Contributing Member
 
Crook's Avatar
 Gallery
LightWave Terragen ZBrush Photoshop After Effects
 
Join Date: Dec 2007
Location: Oxford, UK
Posts: 2,530
Default

Surely someone can come up with an app that can read an image, but when it gets beyond a certain point, it fills in the blurry pixels with procedural content to maintain detail, so you get an infinitely zoomable moon without 22Gb of data?
__________________
Facebook - Andy Crook
Crook is offline   Reply With Quote
Old 30th Jun 2017, 12:43 PM   #7
Starbase1
Registered Member
 
Starbase1's Avatar
 Gallery
Blender LightWave Modo Poser Terragen Vue Gimp Paint Shop Pro Photoshop
 
Join Date: Dec 2007
Location: London
Posts: 12,688
Default

Quote:
Originally Posted by Crook View Post
Surely someone can come up with an app that can read an image, but when it gets beyond a certain point, it fills in the blurry pixels with procedural content to maintain detail, so you get an infinitely zoomable moon without 22Gb of data?
Well, this is a dump from a scientific file designed to revel the maximum amount of real data. I suspect some areas may not have this maxiomum detail - the probes were certainly capable of picking ut up.

And that's the appeal for me - it may well be possible to get convincing detail like you were a few milkes above the surface.
__________________
- - - - - -
My web site: http://nick-stevens.com/
Me on twitter, @runnymonkey, https://twitter.com/runnymonkey

Starbase1 is offline   Reply With Quote
Old 2nd Jul 2017, 11:45 AM   #8
Starbase1
Registered Member
 
Starbase1's Avatar
 Gallery
Blender LightWave Modo Poser Terragen Vue Gimp Paint Shop Pro Photoshop
 
Join Date: Dec 2007
Location: London
Posts: 12,688
Default

Well, a friend (with 128 Gb of memory!) managed to slice me out a chunk, but I'm only seeing it as 2 bit black oe white!

Any suggestions for gettinbg this into LW as a 16 bit displacement map?
Attached Files
File Type: zip Copernicus.zip (10.60 MB, 32 views)
__________________
- - - - - -
My web site: http://nick-stevens.com/
Me on twitter, @runnymonkey, https://twitter.com/runnymonkey

Starbase1 is offline   Reply With Quote
Old 2nd Jul 2017, 06:54 PM   #9
hugojsm
Junior Member
 Gallery
LightWave Photoshop After Effects Premiere
 
Join Date: Jun 2012
Location: Netherlands
Posts: 7
Default Editing large SRTM elevation data

Kinda coincidence I'm trying to render our earth again just at 64K textures but i needed to edit alot of textures again which are about 140k.

There are 2 things going on here. But then again most of them happen at the same time.

1 Your trying to load an amazing/big/large file into an editor (using Photoshop in this example)
2 Your interpreting an elevation data file / SRTM / Shuttle Radar Topography (mission)

So lets start with loading a large elevation file like this.

Your saying you have enough memory but not stating how much.

- Loading a 23GB Tiff image in photoshop needs at least the same amount of RAM + (2GB at start for photoshop) + (your current usage (default 4GB)).
- For every decision you make in photoshop it will need more memory depending on what you do, this is because of the history also and layers.
- Of course lower memory systems should be able to load this kind of file but keep in mind that photoshop is going to use your scratch disk (temporary disk) incase it doesnt have enough memory.
- When your 100% sure you don't need UNDO or STEP Backwards in photoshop go to Edit -> Purge -> All. which will clean all memory. This helps alot editing big files in the process.
- To speed up things the photoshop scratch disk should be your SSD, if you want to keep your SSD alive then this should be your normal disk but takes longer (to explain more read/write on ssd will kill your SSD faster)
- Scratch disk settings can be found in photoshop under Preferences -> Scratch disks -> Choose disk

- You are also not saying what the cpu is of your machine because also this matters alot. (resizing etc)

Now to get back to your elevation file.

You are saying you got a part of the image but your seeing it as black and white. Well to simple explain for example, this is the earth elevation data.

Click image for larger version

Name:	earth_srtm.jpg
Views:	12
Size:	101.5 KB
ID:	84142

You see all black and white. Now i'm not going in to detail how satellites are gathering this data (normally with satellite data you will have way more channels/bands in your image) but for elevation data its quite simple just black/white.

White mostly up. Dark mostly down.
Now why is everything dark? its kinda simple the highest point on earth Mount Everest 8,848 m (29,029 feet) above sea level.
The data in 16/bit goes from highest to lowest. If you compare the highest point to the lowest then on average everything is dark on an elevation file.

Now lets move the white and dark a bit around using the light levels in photoshop.
Click image for larger version

Name:	earth4.jpg
Views:	12
Size:	255.2 KB
ID:	84143

Now lets take a look at your 28gig file.
If you look at your big file its the same but kinda reverse.

This is the original.

Click image for larger version

Name:	test1.jpg
Views:	12
Size:	263.4 KB
ID:	84144

This is with light levels changed.

Click image for larger version

Name:	test1_v1.jpg
Views:	14
Size:	390.8 KB
ID:	84145

All the data is all there you just need to change the contrast (light levels)
Hope you have explanation and how to get it to work, if you still can't i can easily get a portion out of it also just pm me.
hugojsm is offline   Reply With Quote
Old 3rd Jul 2017, 03:13 AM   #10
Starbase1
Registered Member
 
Starbase1's Avatar
 Gallery
Blender LightWave Modo Poser Terragen Vue Gimp Paint Shop Pro Photoshop
 
Join Date: Dec 2007
Location: London
Posts: 12,688
Default

Thanks for taking the time to go into that in depth.

The B&W referred to the smaller sample I attached to my message.
(It's the crater Copernicus, I suspect others will find this handy once in a format more usable with popular 3d software)
__________________
- - - - - -
My web site: http://nick-stevens.com/
Me on twitter, @runnymonkey, https://twitter.com/runnymonkey

Starbase1 is offline   Reply With Quote
Old 3rd Jul 2017, 08:57 AM   #11
John Marchant
Registered Member
 
John Marchant's Avatar
 Gallery
Blender LightWave Terragen Vue Photoshop Premiere
 
Join Date: Nov 2007
Location: Almeria, Spain
Posts: 2,143
Default

This would do the trick, but not free. Maybe talk to them and see if they could test it.


https://www.db-w.com/products/infinimap/about
John Marchant is offline   Reply With Quote
Old 3rd Jul 2017, 02:14 PM   #12
Starbase1
Registered Member
 
Starbase1's Avatar
 Gallery
Blender LightWave Modo Poser Terragen Vue Gimp Paint Shop Pro Photoshop
 
Join Date: Dec 2007
Location: London
Posts: 12,688
Default

Quote:
Originally Posted by John Marchant View Post
This would do the trick, but not free. Maybe talk to them and see if they could test it.


https://www.db-w.com/products/infinimap/about
Thanks John, I remember looking at that one for possibly using the whole of the Blue Marble images...

I've made some progress, the samall sample loads into 3DEM, which gives me a few export options.

Nick
__________________
- - - - - -
My web site: http://nick-stevens.com/
Me on twitter, @runnymonkey, https://twitter.com/runnymonkey

Starbase1 is offline   Reply With Quote
Old 4th Jul 2017, 02:31 AM   #13
John Marchant
Registered Member
 
John Marchant's Avatar
 Gallery
Blender LightWave Terragen Vue Photoshop Premiere
 
Join Date: Nov 2007
Location: Almeria, Spain
Posts: 2,143
Default

Glad its getting somewhere for you nick. For Native LW InfiniMap is the only one i can think of that would allow you to handle that size of image, short of chucking in a skip load more ram, which would mean a new MoBo probably.
John Marchant is offline   Reply With Quote
Old 4th Jul 2017, 02:34 AM   #14
Starbase1
Registered Member
 
Starbase1's Avatar
 Gallery
Blender LightWave Modo Poser Terragen Vue Gimp Paint Shop Pro Photoshop
 
Join Date: Dec 2007
Location: London
Posts: 12,688
Default

Quote:
Originally Posted by John Marchant View Post
Glad its getting somewhere for you nick. For Native LW InfiniMap is the only one i can think of that would allow you to handle that size of image, short of chucking in a skip load more ram, which would mean a new MoBo probably.
I was more interested in getting moderately sized chunks snipped out of it for now - for global views I have a good enough 16 bit displacement map
__________________
- - - - - -
My web site: http://nick-stevens.com/
Me on twitter, @runnymonkey, https://twitter.com/runnymonkey

Starbase1 is offline   Reply With Quote
Old 5th Jul 2017, 07:03 AM   #15
Rigel
Moderator
 
Rigel's Avatar
 Gallery
LightWave Photoshop
 
Join Date: Nov 2007
Location: London (the other one, in Canada)
Posts: 22,545
Default

This is a truly off the wall suggestion, but. . .

Maybe you could write to NASA and ask them if they could send you a file of just the crater you're interested in? Tell them what you're doing with it of course. Who knows, they just might accommodate you.

The tricky part is, I wouldn't know who specifically to contact.
__________________
Those who say it can't be done,
should stop interrupting those of us who are busy doing it!

------------------------------------------------------------
Other Projects

The Universal Joint
Beechcraft 2000 airplane
LadyBug (Firefly class freighter)
Rigel is offline   Reply With Quote

Go Back   Foundation 3D Forums > Techniques, Instruction > 3D
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -6. The time now is 04:48 AM.


2006 - 2013 Foundation 3D