EXSLT - str:tokenize - Implementer Page

Version: 1
Status: new
User Page: index.html
XML Definition: str.tokenize.xml
Function Package: str.tokenize.zip

Function Syntax

node-set str:tokenize(string, string?)

Template Syntax

<xsl:call-template name="str:tokenize">
   <xsl:with-param name="string" select="string" />
   <xsl:with-param name="delimiters" select="string" />?

The str:tokenize function splits up a string and returns a node set of token elements, each containing one token from the string.

The first argument is the string to be tokenized. The second argument is a string consisting of a number of characters. Each character in this string is taken as a delimiting character. The string given by the first argument is split at any occurrence of any of these characters. For example:

str:tokenize('2001-06-03T11:40:23', '-T:')      

Gives the node set consisting of:


If the second argument is omitted, the default is the string '&#x9;&#xA;&#xD;&#x20;' (i.e. whitespace characters). Thus:

str:tokenize('date math str')      

Gives the node set consisting of:


If the second argument is an empty string, the function returns a set of token elements, each of which holds a single character. Thus:

str:tokenize('foo', '')          

Gives the node set consisting of:


The template version of this function returns a result tree fragment consisting of a number of token elements.

An implementation of this extension function in the EXSLT str namespace must conform to the behaviour described in this document.


Built-in support for str:tokenize is available in the following XSLT processors:

ProcessorProcessor VersionImplemented Version
4XSLT, from 4Suite.0.12.0a31
libxslt from Daniel Veillard et al.1.0.191

The following implementations of str:tokenize are available:

LanguageImplemented VersionCreatorDateDownload
EXSLT Function1Jeni Tennison2001-06-03str.tokenize.function.xsl
XSLT Template1Jeni Tennison2001-06-03str.tokenize.template.xsl
Javascript1Chris Bayes2001-06-21str.tokenize.js
Msxsl1Chris Bayes2001-06-21str.tokenize.msxsl.xsl

Use Cases

Use Case Package: str.tokenize.use-cases.zip

The following use cases illustrate the functionality of str:tokenize.

DetailsCalling StylesheetXML DataExpected ResultCategoryTemplate

Change History

Submitted: 2001-06-03
Creator: Jeni Tennison(http://www.jenitennison.com/)

The str:tokenize function splits a string at the occurrences of given characters.
1.12001-06-21Jeni Tennison Added Javascript and MSXML implementations.
1.22001-11-18Jeni Tennison Indicated behaviour when second argument is an empty string; str:tokenize() should return a list of tokens, each containing a single character.
1.32002-08-21Craig Stewart

Added 4XSLT and libxslt implementation to the list.

1.42002-11-12Craig Stewart

Updated 4XSLT version to 0.12.0a3.

http://www.exslt.org/str/functions/tokenize/str.tokenize.html last modified 2002-11-12