Helper<ToolKit> provides several classes and algorithms designed to manipulate ANSI/ISO strings. The class os_substring provides nonintrusive manipulation of ANSI/ISO strings. Regular expressions are supported by the class os_regexp , and through a family of generic algorithms consisting of os_grep , os_grep_if , os_substitute , and os_substitute_copy . Finally, strings can be parsed into tokens using the flexible os_tokenizer class.

Regular Expressions

Helper<ToolKit> contains extensive support for regular expressions. The class os_regexp provides a traditional interface for finding and replacing regular expressions within a string.

This section is not intended to provide instruction in the use of regular expressions. Instead, this section provides examples of the Helper<ToolKit> support for regular expressions.

The following example uses the os_regexp class to search an ANSI/ISO string.

Example <ospace/helper/examples/regexpr1.cpp>


    #include <iostream>
#include <string>
#include <ospace/helper.h>

void
search( const string& str, const char* p )
  {
  string pattern( p );
  os_regexp rx( pattern );

  os_substring range = rx.search( str );
  cout << "string = " << str << endl;
  cout << "pattern = " << pattern << endl;
  cout << "search = ";

  if ( !range.empty() )
    cout << range << endl;
  else
    cout << "Pattern not found." << endl;
  cout << endl;
  }

void
main()
  {
  string str( "Systems<ToolKit> is really cool." );
  search( str, "S[^ ]*s" );
  search( str, "T.*T" );
  search( str, "T.*t" );
  search( str, "S*" );
  }

string = Systems<ToolKit> is really cool.
pattern = S[^ ]*s
search = Systems

string = Systems<ToolKit> is really cool.
pattern = T.*T
search = Pattern not found.

string = Systems<ToolKit> is really cool.
pattern = T.*t
search = ToolKit

string = Systems<ToolKit> is really cool.
pattern = S*
search = S

Since it is limiting to manipulate only a single string at a time, we have developed powerful generic algorithms capable of searching STL containers, IOStreams, and regular C arrays for regular expression patterns.

The following algorithms provide powerful regular expression control over generic containers.

os_grep -Search for strings matching a regular expression pattern.
os_grep_if -Search for strings matching a regular expression pattern using predicate.
os_substitute -Search and replace strings that match a regular expression pattern.
os_substitute_copy -Copy a collection of strings, while searching and replacing strings that match a regular expression pattern.

The following example uses the os_substitute algorithm to perform regular expression substitution on a vector of strings.

Example <ospace/helper/examples/regexpr6.cpp>


    #include <iostream>
#include <algorithm>
#include <vector>
#include <ospace/helper.h>

void
print( const string& s )
  {
  cout << '\t' << s << endl;
  }

void
main()
  {
  // Build a vector of strings.
  vector< string > v;
  v.push_back( "Web<Component> and Web<Component> are C++ libraries." );
  v.push_back( "STL<ToolKit> and Helper<Component> are cool." );
  v.push_back( "JGL is cool." );

  // Display the vector contents.
  cout << "The original vector:" << endl;
  for_each( v.begin(), v.end(), print );

  // Perform regular expression replacements.
  cout << "\nReplace the first occurrence of \"Web\" with \"Systems\":"
       << endl;
  os_substitute
    (
    v.begin(),
    v.end(),
    string( "Web" ),
    string( "Systems" ),
    false                       //First occurence only
    );
  for_each( v.begin(), v.end(), print );
  cout << "\nReplace all occurrences of \"Component\" with \"ToolKit\":"
       <<endl;
  os_substitute
    (
    v.begin(),
    v.end(),
    string( "Component" ),
    string( "ToolKit" ),
    true                        //Global - all occurences
    );
  for_each( v.begin(), v.end(), print );
  cout << "\nReplace \"JGL\" with \"JGL, the Java Generic Library,\":"
       << endl;
  os_substitute
    (
    v.begin(),
    v.end(),
    string( "JGL" ),
    string( "JGL, the Java Generic Library," ),
    false
    );
  for_each( v.begin(), v.end(), print );
  cout << "\nReplace the pattern \" [cC][^.]*\" with \" portable\":"
       << endl;

  os_substitute
    (
    v.begin(),
    v.end(),
    string( " [cC][^.]*" ),
    string( " portable" ),
    true
    );
  for_each( v.begin(), v.end(), print );
  }

The original vector:
        Web<Component> and Web<Component> are C++ libraries.
        STL<ToolKit> and Helper<Component> are cool.
        JGL is cool.

Replace the first occurrence of "Web" with "Systems":
        Systems<Component> and Web<Component> are C++ libraries.
        STL<ToolKit> and Helper<Component> are cool.
        JGL is cool.

Replace all occurrences of "Component" with "ToolKit":
        Systems<ToolKit> and Web<ToolKmt> are C++ libraries.
        STL<ToolKit> and Helper<ToolKit> are cool.
        JGL is cool.

Replace "JGL" with "JGL, the Java Generic Library,":
        Systems<ToolKit> and Web<ToolKit> are C++ libraries.
        STL<ToolKit> and Helper<ToolKit> are cool.
        JGL, the Java Generic Library, is cool.

Replace the pattern " [cC][^.]*" with " portable":
        Systems<ToolKit> and Web<ToolKit> are portable.
        STL<ToolKit> and Helper<ToolKit> are portable.
        JGL, the Java Generic Library, is portable.

Substrings

It is often useful to manipulate a portion of a string independently from the whole string, without the complications of starting positions, offsets, and adjustment values. Helper<ToolKit> provides a substring class capable of extracting and modifying portions of an ANSI/ISO string in a nonintrusive and user-friendly manner.

The following example manipulates portions of two strings using the os_substring class.

Example <ospace/helper/examples/substr1.cpp>


    #include <iostream>
#include <string>
#include <ospace/helper.h>
int
main()
  {
  string str1 = "012345678901234567890123456789";
  string str2 = "abcdefghijklmnopqrstuvwxyz";
  substring substr1( str1, 10, 9 );
  substring substr2( str2, 10, 9 ); // k-s
  substr1 = " cool ";
  cout << str1 << endl;
  cout << substr1 << endl;
  cout << substr2 << endl;
  substr2 = substr1;
  cout << str2 << endl;
  }

0123456789 cool 90123456789
 cool
klmnopqrs
abcdefghij cool tuvwxyz

Tokenizer

Using tokenizer, you can split a string into a vector of tokens based on a variety of criteria. The tokenizer constructor takes up to six parameters.

A string of separators -Any character in the input string matching one of these is interpreted by the tokenizer as a token boundary. The default is "\t".
Allow empty tokens? -If this is true , two consecutive separators are treated as delimiting an empty token. The default is false .
A string of characters to ignore -The tokenizer skips and ignores any character in the input string matching one of these. The default is "".
A string of terminators -Tokenizing halts when a character in the input string is either one of these or a null. A terminator is never placed into a token. The default is "\n".
A string of terminals -Any character in the input string matching one of these is considered to be a single-character token. The default is "".
Include separators? -If true , a separator is counted as the last character in the token it delimits. The default is false .

For information about vectors, consult the Standards User Guide and Reference Manual.

In the following example, a tokenizer is used as a primitive lexical analyzer to parse the start of a C++ header. The first attempt at tokenizing incorrectly identifies "D," and "C:" as tokens because the colon (:) and comma (,) characters are not recognized as individual C++ tokens.

Example <ospace/helper/examples/token1.cpp>


    #include <vector>
#include <iostream>
#include <ospace/helper.h>

void
display_tokens( string str, vector< string >& tokens )
  {
  cout << str << " has " << tokens.size() << " tokens:" << endl;
  vector< string >::const_iterator i;
  for ( i = tokens.begin(); i != tokens.end(); i++ )
    cout << "  \"" << *i << "\"" << endl;
  cout << endl;
  }

void
main()
  {
  const char* str = "class C:   public D, public E\n";

  os_tokenizer tokenizer1;
  vector< string > tokens;
  tokens = tokenizer1.tokenize( str );
  display_tokens( str, tokens );

  os_tokenizer tokenizer2( " ", false, "", "\n", ":,", false );
  tokens = tokenizer2.tokenize( str );
  display_tokens( str, tokens );
  }

class C:   public D, public E
 has 6 tokens:
  "class"
  "C:"
  "public"
  "D,"
  "public"
  "E"

class C:   public D, public E
 has 8 tokens:
  "class"
  "C"
  ":"
  "public"
  "D"
  ","
  "public"
  "E"

Regular Expressions and String Support