I'm trying to use ClosedXml to generate a large sheet (620K rows, 10 columns, CSV version approx 104 MB), loading the data and creating the sheets needs a lot of memory (approx 4 GB) but this is fine. But then when I try to SaveAs() I get an OutOfMemory exception when the process uses approx 9 GB of memory.
I believe the _problem_ is that the DOM approach is being used with the OpenXML SDK. Is there any chance this can be rewritten to use the streaming approach (i.e. using OpenXmlWriter)?
I don't have experience using the OpenXML SDK, but I would be willing to contribute if you can provide some pointers.
The simplest approach (least code impact which would seem to work in my particular situation) I considered was a two phase approach. First use the DOM approach to write an baseline file, then re-open the file using OpenXmlReader and write a replacement WorksheetPart using OpenXmlWriter basically copying what is there but creating a new SheetData in which the rows and their columns are streamed. But like I said I have no real experience is this area so this might be completely wrong.
Comments: I am having the same issue. I have users that have 5GB RAM on 32-bit machines and the large report files they are trying to export throw OutOfMemory exceptions. These files are only about 16MB but it completely chews up their memory during the SaveAs() function. Upgrading the machines to handle it at this point is not an option.
I believe the _problem_ is that the DOM approach is being used with the OpenXML SDK. Is there any chance this can be rewritten to use the streaming approach (i.e. using OpenXmlWriter)?
I don't have experience using the OpenXML SDK, but I would be willing to contribute if you can provide some pointers.
The simplest approach (least code impact which would seem to work in my particular situation) I considered was a two phase approach. First use the DOM approach to write an baseline file, then re-open the file using OpenXmlReader and write a replacement WorksheetPart using OpenXmlWriter basically copying what is there but creating a new SheetData in which the rows and their columns are streamed. But like I said I have no real experience is this area so this might be completely wrong.
Comments: I am having the same issue. I have users that have 5GB RAM on 32-bit machines and the large report files they are trying to export throw OutOfMemory exceptions. These files are only about 16MB but it completely chews up their memory during the SaveAs() function. Upgrading the machines to handle it at this point is not an option.