> Java > StringTokenizer

Java StringTokenizer Class

Last Updated: 28 May, 2022

StringTokenizer is a utility class as well as a legacy class in Java. StringTokenizer was introduced in the JDK 1.0 version and is available in the java.util package.

StringTokenizer Declaration in Java

public class StringTokenizer extends Object implements Enumeration<Object>

The StringTokenizer class is used to break a string into multiple pieces or sub-strings called tokens, It breaks the string values as per the delimiters. The delimiters (the characters that separate tokens) may be whitespaces, tabs, newlines, carriage returns, etc. At token creation time, a set of delimiters can be specified, or on a per-token basis.

A StringTokenizer object internally maintains the current position within the string to be tokenized. Some operations advance this current position past the characters processed. A token is returned by taking a substring of the string that was used to create the StringTokenizer object.

As StringTokenizer is a legacy class so it's recommended to not use this class instead of using another way, for example, suppose you have to split the string then use split() method of the String class or the java.util.regex package.

StringTokenizer Class Constructors Summary

StringTokenizer(String str)

Constructs a string tokenizer for the specified string.

  StringTokenizer st = new StringTokenizer("This is a sample text");
  while (st.hasMoreTokens()) {
    System.out.println(st.nextToken());
  }

StringTokenizer(String str, String delim)

It creates StringTokenizer with specified string and delimiter.

  StringTokenizer st = new StringTokenizer("Java, Spring, MongoDB", ",");  
  while (st.hasMoreTokens()) {  
    System.out.println(st.nextToken());  
  }

StringTokenizer(String str, String delim, boolean returnDelims)

It creates StringTokenizer with specified string, delimiter and returnDelims flag (true/false).
true : delimiter characters are considered to be tokens.
false : delimiter characters serve to separate tokens.

  StringTokenizer st = new StringTokenizer("Java, Spring, MongoDB", ",", true);  
  while (st.hasMoreTokens()) {  
    System.out.println(st.nextToken());  
  }

Java StringTokenizer Methods

Methods	Description
`public int countTokens()`	This method returns the total number of tokens.
`public String nextToken()`	Returns the next token from this string tokenizer.
`public String nextToken (String delim)`	Returns the next token, after switching to the new delimiter set.
`public boolean hasMoreTokens()`	Returns true if and only if there is at least one token in the string after the current position; false otherwise.
`public boolean hasMoreElements()`	Returns true if there are more tokens; false otherwise.
`public Object nextElement()`	Returns the next token in the string.

Now let's see an examples and understand the StringTokenizer class

Example: Using StringTokenizer(String str) Constructor

  import java.util.StringTokenizer;
  public class Example1 {
    public static void main(String[] args) {
    String inputString = "String, StringBuffer, StringBuilder, StringTokenizer";
      
    StringTokenizer tokenizer = new StringTokenizer(inputString);
      while (tokenizer.hasMoreTokens()) {
          System.out.println(tokenizer.nextToken());
      }
    }
  }

Output

String,
StringBuffer,
StringBuilder,
StringTokenizer

Example: Using StringTokenizer(String str, String delim) Constructor

  import java.util.StringTokenizer;
  public class Example2 {
    public static void main(String[] args) {
      String inputString1 = "String, StringBuffer, StringBuilder, StringTokenizer";
  
      System.out.println("Passing Delimiter as ',' ====== ");
      StringTokenizer tokenizer1 = new StringTokenizer(inputString1, ", ");
      while (tokenizer1.hasMoreTokens()) {
          System.out.println(tokenizer1.nextToken());
      }
      
      System.out.println("\nPassing Delimiter as ':' ====== ");
      String inputString2 = "String : StringBuffer : StringBuilder : StringTokenizer";
  
      StringTokenizer tokenizer2 = new StringTokenizer(inputString2, " : ");
      while (tokenizer2.hasMoreTokens()) {
          System.out.println(tokenizer2.nextToken());
      }
    }
  }

Output

Passing Delimiter as ',' ======
String
StringBuffer
StringBuilder
StringTokenizer

Passing Delimiter as ':' ======
String
StringBuffer
StringBuilder
StringTokenizer

Example: Using StringTokenizer(String str, String delim, boolean returnDelims) Constructor

  import java.util.StringTokenizer;
  public class Example3 {
    public static void main(String[] args) {
      String inputStr = "String, StringBuffer, StringBuilder, StringTokenizer";
        
      System.out.println("Passing returnDelims flag as true =====");
      StringTokenizer tokenizer1 = new StringTokenizer(inputStr, ", ", true);
      while (tokenizer1.hasMoreTokens()) {
          System.out.println(tokenizer1.nextToken());
      }
          
      System.out.println("\nPassing returnDelims flag as false =====");
      StringTokenizer tokenizer2 = new StringTokenizer(inputStr, ", ", false);
      while (tokenizer2.hasMoreTokens()) {
          System.out.println(tokenizer2.nextToken());
      }
    }
  }

Output

Passing returnDelims flag as true =====
String
,

StringBuffer
,

StringBuilder
,

StringTokenizer

Passing returnDelims flag as false =====
String
StringBuffer
StringBuilder
StringTokenizer

Example: Using countTokens() method

  import java.util.StringTokenizer;
  public class Example4 {
    public static void main(String[] args) {
      String inputString = "String, StringBuffer, StringBuilder, StringTokenizer";
      StringTokenizer tokenizer = new StringTokenizer(inputString);
      
      // Printing the total number of tokens in the String  
      System.out.println("Total number of Tokens: " + tokenizer.countTokens());    
    }
  }

Output

Total number of Tokens: 4

Example: Using nextToken(String delim) method

  public class Example5 {
    public static void main(String[] args) {
  
      String inputString = "String, StringBuffer, StringBuilder, StringTokenizer";
      StringTokenizer tokenizer = new StringTokenizer(inputString, ", ");
      
      // printing next token of StringTokenizer  
      System.out.println("Next Tokens: " + tokenizer.nextToken());    
    }
  }

Output

Next Tokens: String

That's all guys, hope this Java article is helpful for you.

Happy Learning... 😀

Please share this article on social media to help others.

If you have any query or suggestion regarding this artical please share with us
feedback@javabytechie.com

StringTokenizer - Interview Questions and Answers

Can we pass more than one delimiter to a StringTokenizer?

Ans: Yes. We can pass any number of delimiters as an argument to a StringTokenizer. In order to work this, we have to separate each delimiter by “//” (double forward slash) or directly we can represent them.

Instead of StringTokenizer what is another alternate way to break a String?

We can use split() method of java.lang.String class to split String. String split() method is easier to use and better because it expects a regular expression and returns an array of String that we can further manipulate in the code.