Ticket #218 (closed defect: fixed)
Opened 2007-11-13T15:46:06-06:00
Last modified 2007-11-15T14:29:07-06:00
Fix memory issue with OLE-based formats
| Reported by: | melissa | Owned by: | melissa |
|---|---|---|---|
| Priority: | critical | Milestone: | |
| Component: | bio-formats | Severity: | major |
| Keywords: | Cc: | ||
| Blocked By: | Blocking: |
Description
Zeiss ZVI, FV1000 OIB, ImagePro IPW and SimplePCI all have significant memory issues due to POI. Right now, POI is structured such that the *entire* file is read into memory - this is less than optimal under any circumstances, but becomes really problematic for files > 200 MB.
As far as I can tell, POI keeps a list of all of the data blocks. We should probably replace this with a list of offsets to the data blocks, and read the actual data on demand. This might be a bit slower, but should save lots of memory.
Change History
comment:1 Changed 2007-11-15T14:29:07-06:00 by melissa
- Status changed from new to closed
- Resolution set to fixed
Fixed in r3372. Used the solution described above, which changed POI's API slightly.
For comparison, the overhead for reading an OLE file was (file size) + 50 MB; it's now ~12 MB. Also, there doesn't appear to be a significant speed decrease, which is nice.