java.util

Class StringTokenizer

Implemented Interfaces:
Enumeration<E>

public class StringTokenizer
extends Object
implements Enumeration<E>

The string tokenizer class allows an application to break a string into tokens. The tokenization method is much simpler than the one used by the StreamTokenizer class. The StringTokenizer methods do not distinguish among identifiers, numbers, and quoted strings, nor do they recognize and skip comments.

The set of delimiters (the characters that separate tokens) may be specified either at creation time or on a per-token basis.

An instance of StringTokenizer behaves in one of two ways, depending on whether it was created with the returnDelims flag having the value true or false:

A StringTokenizer object internally maintains a current position within the string to be tokenized. Some operations advance this current position past the characters processed.

A token is returned by taking a substring of the string that was used to create the StringTokenizer object.

The following is one example of the use of the tokenizer. The code:

     StringTokenizer st = new StringTokenizer("this is a test");
     while (st.hasMoreTokens()) {
         System.out.println(st.nextToken());
     }
 

prints the following output:

     this
     is
     a
     test
 

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.

The following example illustrates how the String.split method can be used to break up a string into its basic tokens:

     String[] result = "this is a test".split("\\s");
     for (int x=0; x<result.length; x++)
         System.out.println(result[x]);
 

prints the following output:

     this
     is
     a
     test
 
Since:
JDK1.0
See Also:
StreamTokenizer

Constructor Summary

StringTokenizer(String str)
Constructs a string tokenizer for the specified string.
StringTokenizer(String str, String delim)
Constructs a string tokenizer for the specified string.
StringTokenizer(String str, String delim, boolean returnDelims)
Constructs a string tokenizer for the specified string.

Method Summary

int
countTokens()
Calculates the number of times that this tokenizer's nextToken method can be called before it generates an exception.
boolean
hasMoreElements()
Returns the same value as the hasMoreTokens method.
boolean
hasMoreTokens()
Tests if there are more tokens available from this tokenizer's string.
Object
nextElement()
Returns the same value as the nextToken method, except that its declared return value is Object rather than String.
String
nextToken()
Returns the next token from this string tokenizer.
String
nextToken(String delim)
Returns the next token in this string tokenizer's string.

Methods inherited from class java.lang.Object

clone, equals, extends Object> getClass, finalize, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

StringTokenizer

public StringTokenizer(String str)
Constructs a string tokenizer for the specified string. The tokenizer uses the default delimiter set, which is " \t\n\r\f": the space character, the tab character, the newline character, the carriage-return character, and the form-feed character. Delimiter characters themselves will not be treated as tokens.
Parameters:
str - a string to be parsed.
Throws:
NullPointerException - if str is null
Usages and Demos :

View More Examples of StringTokenizer(String str)
   1:         {
   2:             this.tok = new StringTokenizer("");
   3:             return;
   4:         }

View Full Code Here
   1: 
   2:           typesID = oTypesID != null ? new StringTokenizer(oTypesID, ",") : new StringTokenizer("");
   3:         ...
   4:           objsID = objectsID != null ? new StringTokenizer(objectsID, ",") : new StringTokenizer("");
   5:         ...
   6:           cmpsName = componentsName != null ? new StringTokenizer(componentsName, ",") : new StringTokenizer("");

View Full Code Here
   1:     public boolean validComponent(JComponent jTextField) {
   2:         StringTokenizer st = new StringTokenizer(((JTextField) jTextField)
   3:                 .getText(), ",", true);
   4:         Vector new_signature = new Vector();
   5:         HierarchyProperty hprop = (HierarchyProperty) EditorManager.instance()

View Full Code Here
   1:             if ( allRoles != null && allRoles instanceof String ) {
   2:                 final StringTokenizer st = new StringTokenizer( (String)allRoles, ",");
   3:                 while ( st.hasMoreElements() ) {
   4:                     this.roles.add(st.nextElement());
   5:                 }

View Full Code Here
   1:       tfText_.setText(kei.getAlias()+line);
   2:       StringTokenizer st=new StringTokenizer(" ");
   3:       FuVectorString  vs=new FuVectorString();
   4:       while(st.hasMoreTokens())
   5:         vs.addElement(st.nextToken());

View Full Code Here

StringTokenizer

public StringTokenizer(String str,
                       String delim)
Constructs a string tokenizer for the specified string. The characters in the delim argument are the delimiters for separating tokens. Delimiter characters themselves will not be treated as tokens.

Note that if delim is null, this constructor does not throw an exception. However, trying to invoke other methods on the resulting StringTokenizer may result in a NullPointerException.

Parameters:
str - a string to be parsed.
delim - the delimiters.
Throws:
NullPointerException - if str is null
Usages and Demos :

View More Examples of StringTokenizer(String str,String delim)
   1:         ctxt.generateJavaSource("java.util.StringTokenizer " + stName + " = " +
   2:                 "new java.util.StringTokenizer(" + itemsName + ", " + delimsName + ");");
   3:         
   4:         ctxt.generateJavaSource("int " + tokensCountName + " = " + stName + ".countTokens();");
   5:         if(hasBegin){

View Full Code Here
   1:                 String tmpMsg = result;
   2:                 StringTokenizer tok = new StringTokenizer( "\n", params );
   3:                 boolean first = true;
   4:         ...
   5:             String tmpMsg = result;
   6:             StringTokenizer tok = new StringTokenizer( "\n", _params );
   7:             boolean first = true;
   8:             String line;
   9:             while (tok.hasMoreTokens())

View Full Code Here
   1:     {
   2:         StringTokenizer tok = new StringTokenizer("\" ", " ");
   3:         Assert.assertEquals("\"", tok.nextToken());
   4:         ...
   5:         tok = new StringTokenizer("a", "\"");
   6:         Assert.assertEquals("a", tok.nextToken());
   7:         ...
   8:         tok = new StringTokenizer("\"\"", "\" ");
   9:         Assert.assertEquals(0, tok.countTokens());

View Full Code Here
   1:     StringTokenizer st =
   2:         new StringTokenizer(ISASegment, sElementSeparator);
   3:     for (i=0; i < 17; i++)
   4:         ...
   5:     StringTokenizer st =
   6:         new StringTokenizer(IEASegment, sElementSeparator);
   7:     for (i=0; i < 3; i++)
   8:         ...
   9:     StringTokenizer st =
  10:         new StringTokenizer(GSSegment, sElementSeparator);
  11:     for (i=0; i < 9; i++)

View Full Code Here
   1:         
   2:         StringTokenizer st = new StringTokenizer(FIRST_DOCUMENT_ID , "/");
   3:         int l = st.countTokens();
   4:         String secdocumentid = "";
   5:         for (int i=1; i<l; i++) {

View Full Code Here

StringTokenizer

public StringTokenizer(String str,
                       String delim,
                       boolean returnDelims)
Constructs a string tokenizer for the specified string. All characters in the delim argument are the delimiters for separating tokens.

If the returnDelims flag is true, then the delimiter characters are also returned as tokens. Each delimiter is returned as a string of length one. If the flag is false, the delimiter characters are skipped and only serve as separators between tokens.

Note that if delim is null, this constructor does not throw an exception. However, trying to invoke other methods on the resulting StringTokenizer may result in a NullPointerException.

Parameters:
str - a string to be parsed.
delim - the delimiters.
returnDelims - flag indicating whether to return the delimiters as tokens.
Throws:
NullPointerException - if str is null
Usages and Demos :

View More Examples of StringTokenizer(String str,String delim,boolean returnDelims)
   1:     {
   2:         StringTokenizer strt=new StringTokenizer("I+love+you","+",false);
   3:         while(strt.hasMoreElements())
   4:         System.out.println(strt.nextToken());
   5:     }

View Full Code Here
   1:         
   2:         StringTokenizer st = new StringTokenizer(FIRST_DOCUMENT_ID , "/", true);
   3:         int l = st.countTokens();
   4:         for (int i=1; i<l; i++) {
   5:           st.nextToken();

View Full Code Here
   1:     {
   2:      StringTokenizer StrTkn = new StringTokenizer(DataRefs,"~",false);
   3:      String NxtTkn = new String ();
   4:      int TknCnt,i,VX,NX;
   5:      

View Full Code Here
   1:         
   2:         StringTokenizer st = new StringTokenizer(FIRST_DOCUMENT_ID , "/", true);
   3:         int l = st.countTokens();
   4:         for (int i=1; i<l; i++) {
   5:           st.nextToken();

View Full Code Here
   1:         String DN = cert.getSubjectDN().toString();
   2:         StringTokenizer st = new StringTokenizer(DN, ",");
   3:         StringBuffer finalDN = new StringBuffer();
   4:         while (st.hasMoreTokens()) {
   5:             String nextToken = st.nextToken();

View Full Code Here

Method Details

countTokens

public int countTokens()
Calculates the number of times that this tokenizer's nextToken method can be called before it generates an exception. The current position is not advanced.
Returns:
the number of tokens remaining in the string using the current delimiter set.
See Also:
nextToken()
Usages and Demos :

View More Examples of countTokens()
   1:    public StringTokenizerTest(){
   2:       super( "????StringTokenizer??" );
   3:       Container container = getContentPane();
   4:         ...
   5:                String str=new String( event.getActionCommand() );
   6:                StringTokenizer tokens =
   7:         ...
   8:                   new StringTokenizer(str.toUpperCase());
   9:                outputArea.setText( "Number of elements: " +
  10:         ...
  11:                   tokens.countTokens() + "\nThe tokens are:\n" );

View Full Code Here
   1: 
   2: import java.util.StringTokenizer;
   3: import java.util.Vector;
   4:         ...
   5:     {
   6:         StringTokenizer st = new StringTokenizer(s, sep);        
   7:         ...
   8:         Vector v = new Vector(st.countTokens());
   9:         

View Full Code Here
   1:    public TokenTest(){
   2:       super( "????StringTokenizer??" );
   3:       Container container = getContentPane();
   4:         ...
   5:             public void actionPerformed( ActionEvent event ){      
   6:                StringTokenizer tokens =
   7:         ...
   8:                   new StringTokenizer( event.getActionCommand() );
   9:                outputArea.setText( "Number of elements: " +
  10:         ...
  11:                   tokens.countTokens() + "\nThe tokens are:\n" );

View Full Code Here
   1: 
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:     private String _command;
   6:     private StringTokenizer commandLine;
   7:     private boolean useUIDs;
   8:         ...
   9: 
  10:     public void setCommandLine(StringTokenizer st) {
  11:         commandLine = st;
  12:         ...
  13:     {
  14:         return commandLine.countTokens();

View Full Code Here
   1: 
   2: import java.util.StringTokenizer;
   3: import java.util.regex.Pattern;
   4:         ...
   5:     public IssuerOfPatientIDRules(String rules) {
   6:         StringTokenizer stk = new StringTokenizer(rules, " ;\t\r\n");
   7:         ...
   8:         patterns = new Pattern[stk.countTokens()];
   9:         issuers = new String[patterns.length];

View Full Code Here

hasMoreElements

public boolean hasMoreElements()
Returns the same value as the hasMoreTokens method. It exists so that this class can implement the Enumeration interface.
Specified by:
hasMoreElements in interface Enumeration<E>
Returns:
true if there are more tokens; false otherwise.
Usages and Demos :

View More Examples of hasMoreElements()
   1: 
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:     StringBuffer buf = new StringBuffer();
   6:     for (StringTokenizer e = new StringTokenizer(s); e.hasMoreElements();) {
   7:       if (buf.length() > 0)
   8:     buf.append(' ');
   9:       buf.append((String)e.nextElement());

View Full Code Here
   1: import java.util.Properties;
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:         if (propValue!=null) {
   6:             StringTokenizer tokens = new StringTokenizer(propValue, delim);
   7:             while ( tokens.hasMoreTokens() ) {
   8:         ...
   9:                     tokens.nextToken(),
  10:                     tokens.hasMoreElements() ? tokens.nextToken() : ""
  11:                 );

View Full Code Here
   1: import java.util.Properties;
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:         if (propValue!=null) {
   6:             StringTokenizer tokens = new StringTokenizer(propValue, delim);
   7:             while ( tokens.hasMoreTokens() ) {
   8:         ...
   9:                     tokens.nextToken(),
  10:                     tokens.hasMoreElements() ? tokens.nextToken() : StringHelper.EMPTY_STRING
  11:                 );

View Full Code Here
   1: import java.util.Properties;
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:         if (propValue!=null) {
   6:             StringTokenizer tokens = new StringTokenizer(propValue, delim);
   7:             while ( tokens.hasMoreTokens() ) {
   8:         ...
   9:                     tokens.nextToken(),
  10:                     tokens.hasMoreElements() ? tokens.nextToken() : ""
  11:                 );

View Full Code Here
   1:         
   2:         for (StringTokenizer st = new StringTokenizer(strBuff.toString(), new String(delimiter), true);
   3:         ...
   4:              st.hasMoreElements(); ) {
   5:             String token = st.nextToken();
   6:             if (token.indexOf(keyLeader) >= 0) {
   7:             isKey = true;

View Full Code Here

hasMoreTokens

public boolean hasMoreTokens()
Tests if there are more tokens available from this tokenizer's string. If this method returns true, then a subsequent call to nextToken with no argument will successfully return a token.
Returns:
true if and only if there is at least one token in the string after the current position; false otherwise.
Usages and Demos :

View More Examples of hasMoreTokens()
   1: import java.util.ArrayList;
   2: import java.util.StringTokenizer;
   3: import java.util.Map;
   4:         ...
   5:             if (e instanceof String) {
   6:                 for (StringTokenizer st = new StringTokenizer((String)e); st.hasMoreTokens(); ) {
   7:                     Behavior be = layer.findBehavior(st.nextToken());
   8:                     if (be!=null) be.getLayer().removeBehavior(be);
   9:                 }

View Full Code Here
   1: import java.util.Iterator;
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5: 
   6:             for (StringTokenizer tokenizer = new StringTokenizer(_commands, ","); tokenizer.hasMoreTokens();)
   7:             {
   8:                 command = tokenizer.nextToken().toLowerCase().trim();
   9:                 if (COMMAND_CREATE.equals(command))

View Full Code Here
   1: import java.util.List;
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:     private void parseOptions(String classNameAndOptions, Type record) {
   6:         for (StringTokenizer toker = new StringTokenizer(classNameAndOptions,
   7:             String token = toker.nextToken();
   8:             if (record.className == null)
   9:                 record.className = token;

View Full Code Here
   1: import java.net.URL;
   2: import java.util.StringTokenizer;
   3: import java.util.Vector;
   4:         ...
   5:               {
   6:                 StringTokenizer tokenizer = null;
   7:         ...
   8:                 for(tokenizer = new StringTokenizer(args[i], File.pathSeparator); tokenizer.hasMoreTokens();)
   9:                 {
  10:         ...
  11:                   sb.append(process(opts, file));
  12:                   if(tokenizer.hasMoreTokens())

View Full Code Here
   1: import java.util.Map;
   2: import java.util.StringTokenizer;
   3: import java.util.Vector;
   4:         ...
   5: 
   6:         for (StringTokenizer st = new StringTokenizer(classes, ","); st.hasMoreTokens();) {
   7:             String classname = st.nextToken();
   8:             logger.debug("Will load commands from the " + type + " class " + classname);

View Full Code Here

nextElement

public Object nextElement()
Returns the same value as the nextToken method, except that its declared return value is Object rather than String. It exists so that this class can implement the Enumeration interface.
Specified by:
nextElement in interface Enumeration<E>
Returns:
the next token in the string.
Throws:
NoSuchElementException - if there are no more tokens in this tokenizer's string.
Usages and Demos :

View More Examples of nextElement()
   1: import java.awt.*;
   2: import java.util.StringTokenizer;
   3: import javax.swing.*;
   4:         ...
   5:     removeAll();
   6:     StringTokenizer strtok = new StringTokenizer((String)value, "\r\n");
   7:     setLayout(new GridLayout(strtok.countTokens(), 1));
   8:         ...
   9:     while (strtok.hasMoreElements()) {
  10:       label = new JLabel((String)strtok.nextElement(), JLabel.CENTER);
  11:       LookAndFeel.installColorsAndFont(label, 

View Full Code Here
   1: import java.net.*;
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:     void finger(String s) {
   6:         StringTokenizer st = new StringTokenizer(s, "@", true);
   7:         ...
   8:         String user = (String)st.nextElement();
   9:         ...
  10:         st.nextElement();

View Full Code Here
   1:             while ((s = is.readLine()) != null) {
   2:                 StringTokenizer st = new StringTokenizer(s, "@", true);
   3:         ...
   4:                 String user = (String)st.nextElement();
   5:         ...
   6:                 st.nextElement();
   7:         ...
   8:                 String host = (String)st.nextElement();

View Full Code Here
   1: import java.net.InetAddress;
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:     public void setAsText(String text) throws IllegalArgumentException {
   6:         StringTokenizer tokenizer = new StringTokenizer(text, ",");
   7:         
   8:         ...
   9:         }
  10:         String name = (String) tokenizer.nextElement();
  11:         
  12:         ...
  13:         }
  14:         String addressAsString = (String) tokenizer.nextElement();

View Full Code Here
   1: 
   2:       StringTokenizer toke = new StringTokenizer(pageContext.getRequest().getParameter("inetbasics.template.tableNames").toString(), ",");
   3:         while (toke.hasMoreElements())
   4:         ...
   5:         {
   6:             tables.add((String)toke.nextElement());
   7:       }
   8: 
   9:         try

View Full Code Here

nextToken

public String nextToken()
Returns the next token from this string tokenizer.
Returns:
the next token from this string tokenizer.
Throws:
NoSuchElementException - if there are no more tokens in this tokenizer's string.
Usages and Demos :

View More Examples of nextToken()
   1: 
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:                 if (!line.startsWith("#") && (line.trim().length() > 0)) {
   6:                     StringTokenizer st   = new StringTokenizer(line, "\t");
   7:         ...
   8:                     String folderName    = st.nextToken();
   9:         ...
  10:                     String structureName = st.nextToken();

View Full Code Here
   1: import java.io.Reader;
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:                 String theInfo = readerToString(reader);
   6:                 StringTokenizer st = new StringTokenizer(theInfo, "\n");
   7: 
   8:         ...
   9:                 while (st.hasMoreTokens()) {
  10:                     StringTokenizer stt = new StringTokenizer(st.nextToken(), "\t");
  11:                     Article article = new Article();
  12:         ...
  13:                     article.setArticleNumber(Integer.parseInt(stt.nextToken()));

View Full Code Here
   1: 
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:         int port = -1;
   6:         StringTokenizer tokens = new StringTokenizer(uri, ":");
   7:         ...
   8:         String scheme = tokens.nextToken();
   9:         ...
  10:         tokens = new StringTokenizer(tokens.nextToken(), "@");

View Full Code Here
   1: import java.util.Map;
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:         }
   6:         StringTokenizer nBestTok = new StringTokenizer(decodedNBest, ";");
   7:         if (nBestTok.countTokens() % 3 != 0) {
   8:         ...
   9:         while (nBestTok.hasMoreTokens()) {            
  10:             confidences.add(new Float(nBestTok.nextToken()));
  11:         ...
  12:             utterances.add(nBestTok.nextToken());

View Full Code Here
   1: import java.util.Map;
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:         if ( conceptNames != null && conceptNames.trim().length() > 0 ) {
   6:             StringTokenizer st = new StringTokenizer(conceptNames, "\r\n;");
   7:             List names = new ArrayList();
   8:         ...
   9:             while ( st.hasMoreTokens() ) {
  10:                 st1 = new StringTokenizer( st.nextToken(), ":");
  11:         ...
  12:                 names.add( createCodeDS( st1.nextToken(), st1.hasMoreTokens() ? st1.nextToken():"LN" ) );

View Full Code Here

nextToken

public String nextToken(String delim)
Returns the next token in this string tokenizer's string. First, the set of characters considered to be delimiters by this StringTokenizer object is changed to be the characters in the string delim. Then the next token in the string after the current position is returned. The current position is advanced beyond the recognized token. The new delimiter set remains the default after this call.
Parameters:
delim - the new delimiters.
Returns:
the next token, after switching to the new delimiter set.
Throws:
NoSuchElementException - if there are no more tokens in this tokenizer's string.
NullPointerException - if delim is null
Usages and Demos :

View More Examples of nextToken(String delim)
   1: 
   2: import  java.util.StringTokenizer;
   3: import  java.util.NoSuchElementException;
   4:         ...
   5:   {
   6:     StringTokenizer    st;
   7: 
   8:         ...
   9:   {
  10:     StringTokenizer    st;

View Full Code Here
   1:    boolean nxtIsString=false;
   2:    StringTokenizer st=null;
   3:    String delim=new String(" !\"()*+,-.<=>/");
   4:         ...
   5:       while (st.hasMoreTokens()==true) {
   6:          tk=st.nextToken(delim);
   7:          if (tk.equals(" ")==true) continue;
   8:         ...
   9:          if (tk.charAt(0)=='"') {
  10:            tk=st.nextToken("\"");
  11:         ...
  12:            st.nextToken(delim);

View Full Code Here
   1: import java.util.HashMap;
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:             String type, status, action;
   6:             StringTokenizer htmlCode;
   7: 
   8:         ...
   9: 
  10:                 htmlCode = new StringTokenizer(buffRead.readLine());
  11:                 while (!(htmlCode.hasMoreElements())) {

View Full Code Here
   1: import java.util.Date;
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:             String type, date, status, target, action;
   6:             StringTokenizer htmlCode;
   7: 
   8:         ...
   9: 
  10:                 htmlCode = new StringTokenizer(buffRead.readLine());
  11:                 while (!(htmlCode.hasMoreElements())) {

View Full Code Here
   1: import java.io.*;
   2: import java.util.StringTokenizer;
   3: 
   4:         ...
   5:     public void discardLine(){
   6:         setTokenizer(new StringTokenizer(""));
   7:     }
   8:         ...
   9:     protected String nextToken() throws RuntimeException {
  10:         StringTokenizer t = getTokenizer();
  11:         final String delimiters = getDelimiters();
  12:         ...
  13:         }
  14:         return t.nextToken(delimiters);

View Full Code Here