If you try to run any of the sextante algoritms in gvSIG with a more or less big raster file, an OutOfMemoryException is thrown. When sextante launches an algorithm, the gvSIG binding creates two clases to handle the raster data: one read only from the selected raster layer, and an empty one to create the generated raster data. The first one is using a memory buffer if it is small enough, or a cached buffer in any other case, so it has no problems. But the second one uses always a memory buffer. To solve the error, the same gvSIG raster library API method to create a buffer used in the read only case must be used, so the buffer is created in memory or cached depending on its size. Once this has been solved, performance in comparison to other geoprocessing applications seems not to be good. To enhace the performance, the methods of the gvSIG sextante binding and the gvSIG raster library involved in the reading or writing of each raster cell have been reviewed to check for performance issues. Also, a profiler has been used to look for performance hot spots in runtime. After the performance analysis, the changes to perform are: - In the raster library (libRaster), don't check for each pixel if its line is loaded into the current cached block, but only for each line. For example, for a 1.000x1.000 raster, the check is performed 1.000.000 times, but only 1.000 times are needed. This can be done by storing the last checked line, so if a line to check is the same, for sure it has been loaded in the cache. - In the gvSIG sextante bindings (extGvSIGSextanteBindings), when reading or writing raster pixels, there are range checks, raster data type checks, etc. that have a very small cost, but multiplied by a big number of pixels it becomes very noticeable. Also there are some calls to the gvSIG and raster APIs that are performed for each pixel, and its values don't change in all the process. Those methods may be called only one time by storing its value at the bindings classes level. To perform those performance related changes, there is a change which must be performed first in order to allow them and to simply the bindings implementation: The bindings use to read a raster a Grid, which belongs to the gvSIG raster API and is used to handle the raster buffers in an easier way. But in for the new writable raster, another class (RasterMemoryDriver), implemented in the bindings project, is used. The bindings implementation (gvRasterLayer class) must be changed to use always a Grid.
OperatingSystem | None |
BuildNumber | critical |
SubprojectBuildNumber | Mac System 8.0 |
SubprojectResolveBuildNumber | Mac System 8.5 |
Resolution | Fixed |
Severity | major |
SubprojectName | Sextante |
Component | gvSIG - Sextante |
Version | gvSIG - 1.10.0 |
SubprojectVersion | gvSIG - 1.10.0 |
SubprojectResolveVersion | None |
Has patch | Yes |
Comments
Another OutOfMemory bug has been found related to the opening of big raster files: [#15060]. In this case into the gvSIG raster library.
Uploaded raster library cache optimization to avoid checking if the row of each pixel is loaded in the cache. [gvsig-desktop 34630]
Replace RasterMemoryDriver usage with a gvSIG raster library Grid object, and remove it. Also, add some optimizations while getting or setting a pixel value. [gvsig-desktop 34639]
Small NullPointerException correction added- [gvsig-desktop 34646]
I'm going to set the ticket as fixed, as the main errors related to big raster files seems to be solved. Also the performance seems to be better now. For sure there is more work that can be done, but in that case this ticket may be reopened.