org.exist.storage.dom
Class DOMFile

java.lang.Object
  org.exist.storage.btree.Paged
      org.exist.storage.btree.BTree
          org.exist.storage.dom.DOMFile

All Implemented Interfaces:

Lockable

public class DOMFile

extends BTree

implements Lockable

This is the main storage for XML nodes. Nodes are stored in document order. Every document gets its own sequence of pages, which is bound to the writing thread to avoid conflicting writes. The page structure is as follows: | page header | (tid1 node-data, tid2 node-data, ..., tidn node-data) | node-data contains the raw binary data of the node as returned by org.exist.dom.NodeImpl#serialize(). Within a page, a node is identified by a unique id, called tuple id (tid). Every node can thus be located by a virtual address pointer, which consists of the page id and the tid. Both components are encoded in a long value (with additional bits used for optional flags). The address pointer is used to reference nodes from the indexes. It should thus remain unchanged during the life-time of a document. However, XUpdate requests may insert new nodes in the middle of a page. In these cases, the page will be split and the upper portion of the page is copied to a split page. The record in the original page will be replaced by a forward link, pointing to the new location of the node data in the split page. As a consequence, the class has to distinguish three different types of data records: 1) Ordinary record: | tid | length | data | 3) Relocated record: | tid | length | address pointer to original location | data | 2) Forward link: | tid | address pointer | tid and length each use two bytes (short), address pointers 8 bytes (long). The upper two bits of the tid are used to indicate the type of the record (see org.exist.storage.store.ItemId).

Author:

Wolfgang Meier

Nested Class Summary

protected static class	DOMFile.DOMFilePageHeader
protected class	DOMFile.DOMPage
protected class	DOMFile.OverflowDOMPage This represents an overflow page.
protected static class	DOMFile.RecordPos

Nested classes inherited from class org.exist.storage.btree.BTree

BTree.BTreeFileHeader, BTree.BTreeNode, BTree.BTreePageHeader

Nested classes inherited from class org.exist.storage.btree.Paged

Paged.FileHeader, Paged.Page, Paged.PageHeader

Field Summary

static long	DATA_SYNC_PERIOD
static short	FILE_FORMAT_VERSION_ID
static byte	LOB
static byte	LOG_ADD_LINK
static byte	LOG_ADD_MOVED_REC
static byte	LOG_ADD_VALUE
static byte	LOG_CREATE_PAGE
static byte	LOG_INSERT_RECORD
static byte	LOG_REMOVE_EMPTY_PAGE
static byte	LOG_REMOVE_OVERFLOW
static byte	LOG_REMOVE_PAGE
static byte	LOG_REMOVE_VALUE
static byte	LOG_SPLIT_PAGE
static byte	LOG_UPDATE_HEADER
static byte	LOG_UPDATE_LINK
static byte	LOG_UPDATE_VALUE
static byte	LOG_WRITE_OVERFLOW
static short	OVERFLOW
static byte	RECORD

Fields inherited from class org.exist.storage.btree.BTree

BRANCH, buffers, cache, cacheManager, fileId, growthThreshold, isTransactional, KEY_NOT_FOUND, LEAF, LOG_CREATE_BNODE, LOG_INSERT_VALUE, LOG_SET_PARENT, LOG_UPDATE_PAGE, logManager

Fields inherited from class org.exist.storage.btree.Paged

DELETED, LOG, PAGE_SIZE, UNUSED

Constructor Summary

DOMFile(BrokerPool pool, java.io.File file, CacheManager cacheManager)

Method Summary

long	add(Txn transact, byte[] value) Append a value to the current page.
long	addBinary(Txn transaction, DocumentImpl doc, byte[] value) Store a raw binary resource into the file.
void	addToBuffer(DOMFile.DOMPage page)
boolean	close() Close the underlying files.
void	closeDocument()
boolean	create()
Paged.FileHeader	createFileHeader() createFileHeader must be implemented by a Paged implementation in order to create an appropriate subclass instance of a FileHeader.
protected Paged.Page	createNewPage()
Paged.PageHeader	createPageHeader() createPageHeader must be implemented by a Paged implementation in order to create an appropriate subclass instance of a PageHeader.
java.lang.String	debugPageContents(DOMFile.DOMPage page)
java.lang.String	debugPages(DocumentImpl doc, boolean showPageContents)
protected void	dumpValue(java.io.Writer writer, Value key)
java.util.ArrayList	findKeys(IndexQuery query)
protected DOMFile.RecordPos	findRecord(long p)
protected DOMFile.RecordPos	findRecord(long p, boolean skipLinks) Find a record within the page or the pages linked to it.
protected long	findValue(java.lang.Object lock, NodeProxy node) Find a node by searching for a known ancestor in the index.
java.util.ArrayList	findValues(IndexQuery query) Find matching nodes for the given query.
boolean	flush() Flush all buffers to disk.
Value	get(long p) Retrieve node at virtual address p.
Value	get(NodeProxy node) Retrieve a node described by the given NodeProxy.
Value	get(Value key) Retrieve a node by key
byte[]	getBinary(long pageNum) Return binary data stored with #addBinary(byte[]).
protected DOMFile.DOMPage	getCurrentPage(long p) Retrieve the page with page number p
BufferStats	getDataBufferStats()
short	getFileVersion()
Lock	getLock() Get the active Lock object for this file.
java.lang.String	getNodeValue(NodeProxy proxy, boolean addWhitespace) Retrieve the string value of the specified node.
protected byte[]	getOverflowValue(long pnum)
protected Cache	getPageBuffer()
long	insertAfter(Txn transaction, DocumentImpl doc, long address, byte[] value) Insert a new node after the node located at the specified address.
long	insertAfter(Txn transaction, DocumentImpl doc, Value key, byte[] value) Insert a new node after the specified node.
boolean	open() Open the file.
void	printStatistics()
long	put(Txn transaction, Value key, byte[] value) Put a new key/value pair.
protected void	redoAddLink(AddLinkLoggable loggable)
protected void	redoAddMovedValue(AddMovedValueLoggable loggable)
protected void	redoAddValue(AddValueLoggable loggable)
protected void	redoCreatePage(CreatePageLoggable loggable)
protected void	redoInsertValue(InsertValueLoggable loggable)
protected void	redoRemoveEmptyPage(RemoveEmptyPageLoggable loggable)
protected void	redoRemoveOverflow(RemoveOverflowLoggable loggable)
protected void	redoRemovePage(RemovePageLoggable loggable)
protected void	redoRemoveValue(RemoveValueLoggable loggable)
protected void	redoSplitPage(SplitPageLoggable loggable)
protected void	redoUpdateHeader(UpdateHeaderLoggable loggable)
protected void	redoUpdateLink(UpdateLinkLoggable loggable)
protected void	redoUpdateValue(UpdateValueLoggable loggable)
protected void	redoWriteOverflow(WriteOverflowPageLoggable loggable)
void	remove(Txn transaction, Value key)
void	remove(Txn transaction, Value key, long p)
void	remove(Value key) Physically remove a node.
void	remove(Value key, long p) Physically remove a node.
void	removeAll(Txn transaction, long p) Remove a sequence of pages, starting with the page denoted by the passed address pointer p.
void	removeNode(long p) Physically remove a node.
void	removeNode(Txn transaction, long p)
void	removeOverflowValue(Txn transaction, long pnum)
void	removePage(DOMFile.DOMPage page) Remove the specified page.
void	setCurrentDocument(DocumentImpl doc)
void	setOwnerObject(java.lang.Object obj) The current object owning this file.
protected void	undoAddLink(AddLinkLoggable loggable)
protected void	undoAddMovedValue(AddMovedValueLoggable loggable)
protected void	undoAddValue(AddValueLoggable loggable)
protected void	undoCreatePage(CreatePageLoggable loggable)
protected void	undoInsertValue(InsertValueLoggable loggable)
protected void	undoRemoveEmptyPage(RemoveEmptyPageLoggable loggable)
protected void	undoRemoveOverflow(RemoveOverflowLoggable loggable)
protected void	undoRemovePage(RemovePageLoggable loggable)
protected void	undoRemoveValue(RemoveValueLoggable loggable)
protected void	undoSplitPage(SplitPageLoggable loggable)
protected void	undoUpdateHeader(UpdateHeaderLoggable loggable)
protected void	undoUpdateLink(UpdateLinkLoggable loggable)
protected void	undoUpdateValue(UpdateValueLoggable loggable)
protected void	undoWriteOverflow(WriteOverflowPageLoggable loggable)
protected void	unlinkPages(Paged.Page page) unlinkPages unlinks a set of pages starting at the specified Page.
void	update(Txn transaction, long p, byte[] value) Update the key/value pair where the value is found at address p.
boolean	update(Txn transaction, Value key, byte[] value) Update the key/value pair.

Methods inherited from class org.exist.storage.btree.BTree

addValue, addValue, closeAndRemove, create, createFileHeader, createFileHeader, createFileHeader, createRootNode, dump, findValue, getIndexBufferStats, getRootNode, open, query, query, redoCreateBTNode, redoInsertValue, redoRemoveValue, redoSetParent, redoUpdatePage, redoUpdateValue, remove, remove, removeValue, removeValue, requiresRedo, setRootNode, undoInsertValue, undoRemoveValue, undoUpdateValue

Methods inherited from class org.exist.storage.btree.Paged

backupToStream, exists, getFile, getFileHeader, getFreePage, getPage, getPageSize, hexDump, isOpened, isReadOnly, printFreeSpaceList, reuseDeleted, setFile, setPageSize, unlinkPages, writeValue, writeValue, writeValue

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG_CREATE_PAGE

public static final byte LOG_CREATE_PAGE

See Also:

Constant Field Values

LOG_ADD_VALUE

public static final byte LOG_ADD_VALUE

See Also:

Constant Field Values

LOG_REMOVE_VALUE

public static final byte LOG_REMOVE_VALUE

See Also:

Constant Field Values

LOG_REMOVE_EMPTY_PAGE

public static final byte LOG_REMOVE_EMPTY_PAGE

See Also:

Constant Field Values

LOG_UPDATE_VALUE

public static final byte LOG_UPDATE_VALUE

See Also:

Constant Field Values

LOG_REMOVE_PAGE

public static final byte LOG_REMOVE_PAGE

See Also:

Constant Field Values

LOG_WRITE_OVERFLOW

public static final byte LOG_WRITE_OVERFLOW

See Also:

Constant Field Values

LOG_REMOVE_OVERFLOW

public static final byte LOG_REMOVE_OVERFLOW

See Also:

Constant Field Values

LOG_INSERT_RECORD

public static final byte LOG_INSERT_RECORD

See Also:

Constant Field Values

LOG_SPLIT_PAGE

public static final byte LOG_SPLIT_PAGE

See Also:

Constant Field Values

LOG_ADD_LINK

public static final byte LOG_ADD_LINK

See Also:

Constant Field Values

LOG_ADD_MOVED_REC

public static final byte LOG_ADD_MOVED_REC

See Also:

Constant Field Values

LOG_UPDATE_HEADER

public static final byte LOG_UPDATE_HEADER

See Also:

Constant Field Values

LOG_UPDATE_LINK

public static final byte LOG_UPDATE_LINK

See Also:

Constant Field Values

FILE_FORMAT_VERSION_ID

public static final short FILE_FORMAT_VERSION_ID

See Also:

Constant Field Values

LOB

public static final byte LOB

See Also:

Constant Field Values

RECORD

public static final byte RECORD

See Also:

Constant Field Values

OVERFLOW

public static final short OVERFLOW

See Also:

Constant Field Values

DATA_SYNC_PERIOD

public static final long DATA_SYNC_PERIOD

See Also:

Constant Field Values

Constructor Detail

DOMFile

public DOMFile(BrokerPool pool,
               java.io.File file,
               CacheManager cacheManager)
        throws DBException

Method Detail

getPageBuffer

protected final Cache getPageBuffer()

getFileVersion

public short getFileVersion()

Overrides:

getFileVersion in class Paged

Returns:

setCurrentDocument

public void setCurrentDocument(DocumentImpl doc)

add

public long add(Txn transact,
                byte[] value)
         throws ReadOnlyException

Append a value to the current page. This method is called when storing a new document. Each writing thread gets its own sequence of pages for writing a document, so all document nodes are stored in sequential order. A new page will be allocated if the current page is full. If the value is larger than the page size, it will be written to an overflow page.

Parameters:

value - the value to append

Returns:

the virtual storage address of the value

Throws:

ReadOnlyException

addBinary

public long addBinary(Txn transaction,
                      DocumentImpl doc,
                      byte[] value)

Store a raw binary resource into the file. The data will always be written into an overflow page.

Parameters:

value -

Returns:

getBinary

public byte[] getBinary(long pageNum)

Return binary data stored with #addBinary(byte[]).

Parameters:

pageNum -

Returns:

insertAfter

public long insertAfter(Txn transaction,
                        DocumentImpl doc,
                        Value key,
                        byte[] value)

Insert a new node after the specified node.

Parameters:

key -

value -

Returns:

insertAfter

public long insertAfter(Txn transaction,
                        DocumentImpl doc,
                        long address,
                        byte[] value)

Insert a new node after the node located at the specified address. If the previous node is in the middle of a page, the page is split. If the node is appended at the end and the page does not have enough room for the node, a new page is added to the page sequence.

Parameters:

doc - the document to which the new node belongs.

address - the storage address of the node after which the new value should be inserted.

value - the value of the new node.

Returns:

debugPageContents

public java.lang.String debugPageContents(DOMFile.DOMPage page)

close

public boolean close()
              throws DBException

Description copied from class: Paged

Close the underlying files.

Overrides:

close in class BTree

Throws:

DBException

create

public boolean create()
               throws DBException

Overrides:

create in class Paged

Throws:

DBException

createFileHeader

public Paged.FileHeader createFileHeader()

Description copied from class: Paged

createFileHeader must be implemented by a Paged implementation in order to create an appropriate subclass instance of a FileHeader.

Overrides:

createFileHeader in class BTree

See Also:

Paged.createFileHeader()

createNewPage

protected Paged.Page createNewPage()

unlinkPages

protected void unlinkPages(Paged.Page page)
                    throws java.io.IOException

Description copied from class: Paged

unlinkPages unlinks a set of pages starting at the specified Page.

Overrides:

unlinkPages in class Paged

Parameters:

page - The starting Page to unlink

Throws:

java.io.IOException - if an Exception occurs

createPageHeader

public Paged.PageHeader createPageHeader()

Description copied from class: Paged

createPageHeader must be implemented by a Paged implementation in order to create an appropriate subclass instance of a PageHeader.

Overrides:

createPageHeader in class BTree

See Also:

Paged.createPageHeader()

findKeys

public java.util.ArrayList findKeys(IndexQuery query)
                             throws java.io.IOException,
                                    BTreeException

Throws:

java.io.IOException

BTreeException

findValue

protected long findValue(java.lang.Object lock,
                         NodeProxy node)
                  throws java.io.IOException,
                         BTreeException

Find a node by searching for a known ancestor in the index. If an ancestor is found, it is traversed to locate the specified descendant node.

Parameters:

lock -

node -

Returns:

Throws:

java.io.IOException

BTreeException

findValues

public java.util.ArrayList findValues(IndexQuery query)
                               throws java.io.IOException,
                                      BTreeException

Find matching nodes for the given query.

Parameters:

query - Description of the Parameter

Returns:

Description of the Return Value

Throws:

java.io.IOException - Description of the Exception

BTreeException - Description of the Exception

flush

public boolean flush()
              throws DBException

Flush all buffers to disk.

Overrides:

flush in class BTree

Returns:

Description of the Return Value

Throws:

DBException - Description of the Exception

printStatistics

public void printStatistics()

Overrides:

printStatistics in class BTree

getDataBufferStats

public BufferStats getDataBufferStats()

get

public Value get(Value key)

Retrieve a node by key

Parameters:

key -

Returns:

Description of the Return Value

get

public Value get(NodeProxy node)

Retrieve a node described by the given NodeProxy.

Parameters:

node - Description of the Parameter

Returns:

Description of the Return Value

get

public Value get(long p)

Retrieve node at virtual address p.

Parameters:

p - Description of the Parameter

Returns:

Description of the Return Value

getOverflowValue

protected byte[] getOverflowValue(long pnum)

removeOverflowValue

public void removeOverflowValue(Txn transaction,
                                long pnum)

getCurrentPage

protected final DOMFile.DOMPage getCurrentPage(long p)

Retrieve the page with page number p

Parameters:

p - Description of the Parameter

Returns:

The currentPage value

closeDocument

public void closeDocument()

open

public boolean open()
             throws DBException

Open the file.

Returns:

Description of the Return Value

Throws:

DBException - Description of the Exception

put

public long put(Txn transaction,
                Value key,
                byte[] value)
         throws ReadOnlyException

Put a new key/value pair.

Parameters:

key - Description of the Parameter

value - Description of the Parameter

Returns:

Description of the Return Value

Throws:

ReadOnlyException

remove

public void remove(Value key)

Physically remove a node. The data of the node will be removed from the page and the occupied space is freed.

remove

public void remove(Txn transaction,
                   Value key)

removeNode

public void removeNode(long p)

Physically remove a node. The data of the node will be removed from the page and the occupied space is freed.

Parameters:

p -

removeNode

public void removeNode(Txn transaction,
                       long p)

remove

public void remove(Value key,
                   long p)

Physically remove a node. The data of the node will be removed from the page and the occupied space is freed.

remove

public void remove(Txn transaction,
                   Value key,
                   long p)

removePage

public void removePage(DOMFile.DOMPage page)

Remove the specified page. The page is added to the list of free pages.

Parameters:

page -

removeAll

public void removeAll(Txn transaction,
                      long p)

Remove a sequence of pages, starting with the page denoted by the passed address pointer p.

Parameters:

transaction -

p -

debugPages

public java.lang.String debugPages(DocumentImpl doc,
                                   boolean showPageContents)

getLock

public final Lock getLock()

Get the active Lock object for this file.

Specified by:

getLock in interface Lockable

See Also:

Lockable.getLock()

setOwnerObject

public final void setOwnerObject(java.lang.Object obj)

The current object owning this file.

Parameters:

obj - The new ownerObject value

update

public boolean update(Txn transaction,
                      Value key,
                      byte[] value)
               throws ReadOnlyException

Update the key/value pair.

Parameters:

key - Description of the Parameter

value - Description of the Parameter

Returns:

Description of the Return Value

Throws:

ReadOnlyException

update

public void update(Txn transaction,
                   long p,
                   byte[] value)
            throws ReadOnlyException

Update the key/value pair where the value is found at address p.

Parameters:

p - Description of the Parameter

value - Description of the Parameter

Throws:

ReadOnlyException

getNodeValue

public java.lang.String getNodeValue(NodeProxy proxy,
                                     boolean addWhitespace)

Retrieve the string value of the specified node.

Parameters:

proxy -

Returns:

findRecord

protected DOMFile.RecordPos findRecord(long p)

findRecord

protected DOMFile.RecordPos findRecord(long p,
                                       boolean skipLinks)

Find a record within the page or the pages linked to it.

Parameters:

p -

Returns:

redoCreatePage

protected void redoCreatePage(CreatePageLoggable loggable)

undoCreatePage

protected void undoCreatePage(CreatePageLoggable loggable)

redoAddValue

protected void redoAddValue(AddValueLoggable loggable)

undoAddValue

protected void undoAddValue(AddValueLoggable loggable)

redoUpdateValue

protected void redoUpdateValue(UpdateValueLoggable loggable)

undoUpdateValue

protected void undoUpdateValue(UpdateValueLoggable loggable)

redoRemoveValue

protected void redoRemoveValue(RemoveValueLoggable loggable)

undoRemoveValue

protected void undoRemoveValue(RemoveValueLoggable loggable)

redoRemoveEmptyPage

protected void redoRemoveEmptyPage(RemoveEmptyPageLoggable loggable)

undoRemoveEmptyPage

protected void undoRemoveEmptyPage(RemoveEmptyPageLoggable loggable)

redoRemovePage

protected void redoRemovePage(RemovePageLoggable loggable)

undoRemovePage

protected void undoRemovePage(RemovePageLoggable loggable)

redoWriteOverflow

protected void redoWriteOverflow(WriteOverflowPageLoggable loggable)

undoWriteOverflow

protected void undoWriteOverflow(WriteOverflowPageLoggable loggable)

redoRemoveOverflow

protected void redoRemoveOverflow(RemoveOverflowLoggable loggable)

undoRemoveOverflow

protected void undoRemoveOverflow(RemoveOverflowLoggable loggable)

redoInsertValue

protected void redoInsertValue(InsertValueLoggable loggable)

undoInsertValue

protected void undoInsertValue(InsertValueLoggable loggable)

redoSplitPage

protected void redoSplitPage(SplitPageLoggable loggable)

undoSplitPage

protected void undoSplitPage(SplitPageLoggable loggable)

redoAddLink

protected void redoAddLink(AddLinkLoggable loggable)

undoAddLink

protected void undoAddLink(AddLinkLoggable loggable)

redoUpdateLink

protected void redoUpdateLink(UpdateLinkLoggable loggable)

undoUpdateLink

protected void undoUpdateLink(UpdateLinkLoggable loggable)

redoAddMovedValue

protected void redoAddMovedValue(AddMovedValueLoggable loggable)

undoAddMovedValue

protected void undoAddMovedValue(AddMovedValueLoggable loggable)

redoUpdateHeader

protected void redoUpdateHeader(UpdateHeaderLoggable loggable)

undoUpdateHeader

protected void undoUpdateHeader(UpdateHeaderLoggable loggable)

dumpValue

protected void dumpValue(java.io.Writer writer,
                         Value key)
                  throws java.io.IOException

Overrides:

dumpValue in class BTree

Throws:

java.io.IOException

addToBuffer

public final void addToBuffer(DOMFile.DOMPage page)