[XSL-LIST Mailing List Archive Home] [By Thread] [By Date]

RE: [xsl] Huge XML data files >> XSLT

---

---

> 
> I am extrememely sorry to ask such a novice question. I have 
> an xml file (around 150MB) : ...
> 
> Which I am transforming to another XML using XSLT (say 
> pubids.XSL) ...
> 
> eventually I am doing all this so that I can use XMLDBMS to 
> import the data in MySQL
> 
> Everything is working fine when I have small data sheets.
> 
> But it gives outofmemory error even if I increase the heap 
> size. Please suggest some way where I can perform my 
> operation without getting this annoying outofmemory error.

I think you should write an application that loads the data directly
into a database, and then manipulate it further using SQL. Databases are
designed to handle these kinds of data volumes, XSLT isn't.

I don't know what tools MySQL offers for data loading, but it's easy
enough to write as a Java application using SAX and JDBC.

If you don't want to write Java, you could do it using two extensions
offered by Saxon: the saxon:preview extension, which allows you to
process each <article> or <book> element as soon as it is read, without
constructing a source tree, and the SQL extension, which allows you to
update a SQL database directly from the stylesheet.

Alternatively, you could explore STX, which is a streaming XML
transformation language: see http://stx.sourceforge.net/

Michael Kay
Software AG
home: Michael.H.Kay@xxxxxxxxxxxx
work: Michael.Kay@xxxxxxxxxxxxxx 


 XSL-List info and archive:  http://www.mulberrytech.com/xsl/xsl-list

---