This section is of little use for anyone except for programmers willing to contribute to the development of Epos or going to modify its source code. It is also not trying to become a beginner's guide to Epos. Anyway, if you are personally missing something here or elsewhere, tell me and I may add it; that will become almost the only source of progress in this section of documentation. The section may also slowly become outdated due to lack of interest, except for the subsection on portability.
Overall coding priorities, approximately in order of decreasing precedence:
class parser, unit, rules, text and maybe a few others are isolated classes
that take no advantage from inheritance. The reason for the class-oriented design
is just a matter of code readability and decomposition in this case.
simpleparserThis class takes some input (such as a plain ASCII or STML text) and then can be
used in conjunction with the class unit constructor to build the
text structure representation. Its purpose is to
identify the Latin text tokens (usually ASCII characters, but some traditional
tokens like "..." would be difficult to identify later, as well as numerous other
context dependent uses of "."). The parser also identifies the level of description
which corresponds to the token and this is the information needed by the class
unit constructor to correctly build the
TSR. In this process,
the parser skips over any empty units, that is, units that contain no phones
(simple letters) at all.
Note that it is unnecessary and counterproductive to distinguish between homographic tokens used at the same level of description here; such intelligence can be handled more flexibly by the language dependent rules. In fact, they tend to be usually language dependent. The parser only avoids losing information (through empty unit deletion) by the minimum necessary tokenization.
The STML parser is still unimplemented.
unitThis class is the fundamental element of the
text structure representation. Its methods are listed in
unit.h. Every object of this type represents a single
text unit. Every unit includes pointers to its immediate
container, and to its contents. The contents are organized in a
bidirectional linked list; pointers to the head and tail units
of this lists are stored in the unit. These links, i.e. prev
and next, also serve to locate the neighboring units; they may be
NULL, indicating that this is the first/last unit in the
immediate container. For most uses, these pointers are not suitable
to be used directly; the Prev and Next methods find the
neighbor, even if a higher level boundary lies in between. It is also
possible to mark a unit as a scope one. In this case, the Next
and Prev methods will be unable to cross its boundary from inside
out (they will return NULL if this is attempted). If you need to
modify the TSR directly, you will benefit from calling unit::sanity
occasionally. This method checks the TSR structure near the unit which has
called it and will report a severe error, if an invariant is violated,
thus saving you from misguided error messages or crashes later.
To extract the prosodic information from a TSR, call the effective
method. It will combine the prosodic adjustments present at all the levels
of description above the current unit.
textThis class represents a logical line-oriented text file. It handles things like the @include directive, backslash-escaped special characters, initial whitespace and comment stripping. It is used for rule files, configuration files, and also for the dictionaries.
fileThis class represents a physical data file. Its main purpose is to cache and
share files repeatedly needed by Epos. The claim function (to be found in
interf.cc) should be used for opening the file (or only sharing an existing
copy if the file is already open) and reading the data out of the file. The
unclaim function is called separately for every claim call whenever
the file is no more needed.
Any code which uses this class should never extract the data member out of it and
use it independently, even if the class itself remains claimed. This is because
if the content of the file has changed, the data in memory will be reallocated and
re-read upon the next call to claim or possibly even sooner. This may cause
invalidation of the original data member at any point of a control switch
to another Epos agent. It is possible to call reclaim at any time to force
re-reading any file if its time stamp has changed.
hashclass hash is derived from class hash_table<char,char>.
The hash_table template is a generic hash table, keys and associated
data items being its class parameters. This implementation uses balanced (AVL)
trees to resolve collisions and is able to adjust (rehash) itself
when it gets one is left for the remaining 25%.)
The repeat count must be a positive integer. You can not use this feature just after conditional rules, because repeated rules are not counted as a single rule for syntactic purposes:
if $something
2x regress 0>x(!_!) #...wrong!
You should rewrite this to
if $something
{
2x regress 0>x(!_!)
}
Huge integers (like one million) are disallowed. This is because the current implementation needs a few bytes of memory (one pointer) per every repetition.
The conditional rules execute the following rule if and only if a condition is met. The condition is specified as the parameter, the following (conditioned) rule is given on a separate line (or lines, if e.g. a composite rule follows). (Comments, whitespace and empty lines may intervene as usual.) It is not syntactically necessary to indent the conditioned rules with whitespace, but it is strongly recommended for readability.
The conditioned rule is syntactically considered to be a part of the conditional rule.
inside Apply a rule or a block of rules within certain units only. The parameter is a list of values at the scope level, wherein the following rule should be applied; the except operator may be used.
Every unit (a sentence, for example), which fulfills the criterion,
is processed separately, therefore the scope of the following rule may
be at most that of the inside rule itself.
Example:
if phr_break
{
regress 0>\#(!_0) colon
inside \# phone
{
contour t/-65 phone
}
}
This example takes action only if the phr_break variable
is set. The action is to insert a hash character (representing
a pause) to the phone level at the end of every colon, and to
affect pro prosodic values of the new character, so that the pause
is sufficiently short. Notice the necessary escaping of the hash
character so as not to confuse it with a comment-out character.
nearApply a rule or a block of rules within units which contain at least one of the specified units. The parameter is a list of values at the target level, which are looked up in a unit of the scope level; the except operator may be used. If an occurence is found, the following rule gets applied to the scope level unit.
If the parameter begins with an asterisk, the asterisk is treated as an except operator and the test is negated. In other words, the following rule gets applied, if every target level unit contained meets the set description with the leading asterisk ignored. You can combine asterisk and an extra except operator to get tests of the "contains no characters of this class" type.
Example:
near *!$vowel
{
regress $lowercase>$uppercase(!_!)
subst spellout.dic
}
A fragment of this kind can be used to spell out all
words which contain no vowels (and are thus supposedly
unpronounceable). The referenced dictionary spellout.dic
should contain the spelled out equivalents for each upper case
letter. The shift of the word to the upper case may look
puzzling, but it is actually only a technical trick to
prevent the spell-out phrases (which are supposedly listed
in lower case) to be spelled out themselves.
Likewise,
near *$vowel
near $vowel
near !$vowel
with Apply a rule or a block of rules for listed units. In contrast with the preceding rule type, this refers not only to the token at the scope level (such as space), but to the whole structure (such as the string of phones delimited by the space).
The parameter is a dictionary filename or a quoted dictionary; it should list the strings subject to the following rule, such as special words. All the details concerning the syntax of the parameter are exactly the same as with other dictionary oriented rules and a simple example is given at the raise rule.
(Advanced users: replacers can be specified in the dictionary and they will be used to replace the replacee as with any other dictionary-oriented rule, but the replacement process will not be iterated.)
The parameter can optionally be prefixed by an exclamation mark, in which case the subordinate rule will be applied exactly to those units which did not match instead of those which did.
An example of how to apply a block of rules to all words except the words "exception" and "resistant":
with !"exception resistant" word
{
...
}
ifApply a rule or a block of rules only if a condition (given by the parameter) is met. The condition must currently be specified as a boolean voice configuration option (possibly a soft option) or its negation (i.e. prefixed with an exclamation mark).
Example:
if !colloquial
{
...
}
The rules within the block will be applied only if the colloquial option is not set.
This if rule inherits its scope from its parent rule
if not specified explicitly.
Again, the scope of a subordinate rule may not be larger than that of
the if rule itself.
regexRegular expression substitution. The parameter is of the form
/regular_expression/replacement/. This rule type is similar to subst
with only one dictionary item, but it is way more powerful and more arcane;
its use is not intended for end wizards nor trivial tasks.
For a regular expressions' overview, UNIX users can consult e.g. the grep
manual page, whereas Windows users can telnet to a nearby UNIX machine and
write man grep there.
Epos uses the extended regular expression syntax with the following difference:
in "regular" regular expressions, parentheses match themselves, while
the open group and close group operators are \( and \), respectively.
As we use groups heavily and next to none real parentheses, we decided
to do it the other way round. Also, sed users may be surprised
by the iterative behavior of the regex rule type in Epos.
The replacement may contain escape sequences referring to the match of
the n-th group within the regular expression: \1 to \9.
\0 represents the entire match, but this is probably unusable under the
current design, as this would cause an infinite substitution loop.
In order to use this type of rule, you need to have the rx or regex
library already installed and have WANT_REGEX enabled in common.h.
This is because we don't actually implement the regex parsing stuff; we leave it
to your OS libraries. In case you don't have such libraries installed, we use
the glibc implementation (rx.c in the Epos distribution).
Note that if your system doesn't support locale setting nor provides
a usable regex library, you can't use named character classes such
as [:upper:] in your regular expressions. This is the case
on Windows CE.
debug Debugging information during the application of the rules. Scope and target are ignored, the parameter is parsed lazily.
Parameter "elem": dump the current state of the text being processed
Parameter "pause": wait until keypress