I'm trying to use ClosedXml to generate a large sheet (620K rows, 10 columns, CSV version approx 104 MB), loading the data and creating the sheets needs a lot of memory (approx 4 GB) but this is fine. But then when I try to SaveAs() I get an OutOfMemory exception when the process uses approx 9 GB of memory.
I believe the _problem_ is that the DOM approach is being used with the OpenXML SDK. Is there any chance this can be rewritten to use the streaming approach (i.e. using OpenXmlWriter)?
I don't have experience using the OpenXML SDK, but I would be willing to contribute if you can provide some pointers.
The simplest approach (least code impact which would seem to work in my particular situation) I considered was a two phase approach. First use the DOM approach to write an baseline file, then re-open the file using OpenXmlReader and write a replacement WorksheetPart using OpenXmlWriter basically copying what is there but creating a new SheetData in which the rows and their columns are streamed. But like I said I have no real experience is this area so this might be completely wrong.
I believe the _problem_ is that the DOM approach is being used with the OpenXML SDK. Is there any chance this can be rewritten to use the streaming approach (i.e. using OpenXmlWriter)?
I don't have experience using the OpenXML SDK, but I would be willing to contribute if you can provide some pointers.
The simplest approach (least code impact which would seem to work in my particular situation) I considered was a two phase approach. First use the DOM approach to write an baseline file, then re-open the file using OpenXmlReader and write a replacement WorksheetPart using OpenXmlWriter basically copying what is there but creating a new SheetData in which the rows and their columns are streamed. But like I said I have no real experience is this area so this might be completely wrong.