 |
 |
 |
| Department of Theoretical and Applied Linguistics, School of English Language and Literature, Aristotle University of Thessaloniki. | ||
People
 Dr.
Katerina NicolaidisAnna
SfakianakiEleni
TsiartsioniVicky
PapachristouProf.
Eleftherios PanagopoulosResearch & Projects |
Anna
Sfakianaki
Introduction Box of Tricks is developing a workstation that provides real-time visual display of acoustic information for children in need of assistance with various aspects of speech production. During the process of learning speech, children with normal hearing follow a product-oriented approach. They discover how to control their speech organs through reference to acoustic speech signals. In this way they develop the ability to generate all the acoustic effects occurring in speech. Naturally, this process is problematic for speech impaired people. In traditional speech therapy a process-oriented approach is generally used; the speech therapist gives instructions on how to use the speech organs while forming sounds. Nevertheless, during normal speech development, children never receive instructions on how to move or where to place their speech articulators. Instead of
the process-oriented
approach, or to supplement it, Box of Tricks hopes to offer a
product-oriented
one. In speech communication it is not the process of the articulation
that is
important, but the quality of the produced sound by which the
information is
transmitted to the other person. In Box of Tricks -developed for
hearing-impaired children, the produced sound is measured and
visualised. The
user discovers how to control his or her speech organs by comparing the
visual
patterns (speech pictures) of the normal acoustic speech signal with
the
defective one. Additionally, the acoustic pre-processing of the system
uses a
special filter bank imitating the filtering characteristics of the
inner ear.
So the speech picture should be much more similar to the perceived one
than a
simple bank of traditional filters, or FFT spectra. The components of the system The system
consists of two
basic parts. The first part consists of a language-independent editor
and
measuring system which is used to construct the modules for all SPECO
languages. This language-independent editor can be adapted to any
European
language. The second part consists of language-dependent speech
databases. The
participating languages are English, Hungarian, Slovenian and Swedish,
thus
there are four reference speech databases, which the system uses in
order to
make a decision about the microphone input. The Child Speech Database Each language has two databases: the reference-speaker database and the multi-speaker database. The four language versions are divided into two packages: the fricative and affricate support and the vowel support. Regarding the English version, the fricative and affricate support includes the sibilants s, z, S, Z and the affricates tS, and dZ. The vowel support includes the five long vowels i:, 3:, A:, O: and u:, and the six short vowels I, e, Q, {, V and U (symbols are in SAMPA). The fricative and affricate support was recorded with our reference speaker, Charlie, when he was eight years old, and the vowel support was recorded about a year later. All recordings were carried out in the sound-deadened recording room in the speech lab at the university of Reading, using the special editor incorporated in the SPECO system. Each utterance was recorded three times and the best one was saved and chosen to appear as the reference example in the exercises. The reference database was segmented using a special application within the SPECO editor. The reference examples were segmented so as to feed the system with information about the normal range of each phoneme and to demonstrate the arbitrary limits of each phoneme in the exercise window to assist the speech therapist and the client in training. The multi-speaker database contains a portion of the reference material. 36 children aged between 7 and 11 were recorded. Each recording session took approximately 8-12 minutes, depending mostly on how fast the child could read the utterances from the cards. The speakers were selected from three different schools. Two of these schools are situated in or near Reading and the third one is in a suburb of London. It may be worth noting that some children had problems articulating certain fricative and affricate sounds, most commonly [Z] and [dZ], especially in isolation. There were also some articulation problems concerning the sounds [r] and [T]. The multi-speaker database was also segmented but this time using software (WASP) not incorporated in the editor itself. Both
databases have been used
to establish norms which guide the teaching or remediation process. The
segmented material was used in the construction of fricative and vowel
spectra
-“spreadlines”, as we call them, and determined the allowed spectral
deviation.
The spreadlines are constructed for each language separately and
constitute the
actual background of the exercise.
The concept
of the SPECO system
is to visualise speech at a low level of speech processing and to let
clients
use their high level information processing ability to work on this.
Teaching
children how to obtain information from speech pictures is more
preferable to
giving articulation instructions. A detailed examination has been
prepared to
decide what scale of loudness, pitch contour, spectral distribution,
etc.,
gives the most informative visual presentation (speech pictures) of
these
parameters. How can we draw children’s attention to the areas of
maximum energy
in the spectrogram? How can we encourage them to use correct loudness
and
intonation levels? How can children recognise if their rhythm is
appropriate
etc.? Generally we use different amusing background drawings to help
children
find the important parts of the speech pictures. First of all, each
phoneme is
assigned its own symbolic picture so that the child very quickly find
out which
are the significant parts of the screen (Figure 1). Figure 1 The top picture shows typical cochleagrams of the English fricatives and affricates trained by the system. Each sound corresponds to a particular drawing (bottom picture) so that the client can make the necessary association when looking at the speech picture. For example, the correct production of an [s] (which is symbolised with a snake) would cover the most part of the eggs with dots.
Some
examples
of types of
speech pictures are the following: energy changing with time (Figure
2); pitch;
voiced - unvoiced detection; intonation; spectrum; spectrogram
(cochleagram);
spectrogram differences.  Figure 2 By saying pi pi pi, the child must make the yellow ball jump over the heads of the worms with the appropriate rhythm. The system is based on up-to-date technology, but we follow the steps of traditional speech therapy in both modules. These are sound preparation, sound development, followed by training in words and automation (meaning the achievement of a reliable production not requiring further instruction). At the stage sound preparation children are trained to pay the necessary attention to the screen. They start to familiarise themselves with the way curves form on the screen according to sound energy and the position of the speech organs. There is the possibility to train the adjustment of different speech parameters: loudness, rhythm, spectrum, pitch, voicing, intonation. In sound development we start with the forming of individual phonemes. This stage includes working with articulation pictures, isolated pronunciation practice and syllable training. The articulation pictures (Figure 3) show the child which is the right position of all the organs (mouth, tongue, teeth etc.) that play a role in sound production. After teaching the correct articulation, children attempt to produce sustained sounds. The change of the energy measured in each frequency band is visible on the screen. The form of the distribution lines of the correctly pronounced phoneme is different in each case and characterises the phoneme itself (Figure 4). This exercise has three levels of difficulty. At the easiest level the stripe (distribution lines) is wider and the deviation is expected to be considerable, whereas at the most difficult level the stripe becomes narrower and the deviation should be small. The three levels of difficulty exist in both the fricative and the vowel support, but in vowel support the differences among the levels are not as substantial.  Figure 4 The spectrum of the English fricative [S] presented as a speech picture by the program. The objective is to produce and sustain a line within the limits of the “green field”. For syllable training, the vocabulary contains sound sequences constructed so that the phonemes being practised occur in different positions and contexts. These syllables appear on the screen in the form of cochleagrams. For the English fricative and affricate support, fricatives and affricates are presented in CV, VCV, VC and VC-VC-VC position and connected with the five long vowels. Whereas the English vowel support contains all vowels in syllables along with front stops, like [p, t and b]. The order of presentation of sound sequences could be important, so we grade those from the easier pronunciations to the more difficult ones. In this exercise, the reference syllable is demonstrated on the upper half of the screen, while the syllable the client produces appears in the bottom half of the screen (Figure 5). The client attempts to match his picture with the reference one as closely as possible. Figure 5 The reference syllable [3:s] appears on the top half of the screen and the client’s production below. The blue dots correspond to the vowel and the red dots to the fricative. The aim is to cover as much of the eggs as possible with red dots and leave the snake uncovered. In the training in words the grouping of words is different in fricative support and vowel support. In fricative support all phonemes are presented in initial, medial and final position in words. In vowel support all phonemes occur in one-syllable words and in words of two or more syllables. Again the upper half of the screen shows the cochleagram of the reference word and the client has to produce the same word so as to fill in the right parts of the bottom half of the screen (Figure 6). Figure 6 The reference speech picture (cochleagram of the English word ‘kitchen’) above and the client’s production below. The phoneme trained here is [tS] word-medially and its symbolic picture is a station (for the closure) a train (for the lower part of the cochleagram) and its smoke (for the actual release). The objective is to cover most of the smoke (release of [tS]) with red dots and leave the station and the train clear. The automation (or “continuity” for the English version of the system) consists of two parts: contrast pairs and phrases. These exercises work on the basis of cochleagrams as well. The contrast pairs are presented to the child to show the differences between the speech pictures of two phonemes in similar words. For example one of the word pairs chosen to train the phoneme /z/ word-initially is “zip-dip”. The phrases contain the trained phoneme at least once and they are specially designed and graded from simple and short to complex and longer ones. Our aim in the therapy is to reach that speech level at which the client speaks correctly without having to concentrate on the articulation. Therefore, besides the practice with phrases, we have included a category called “free exercise”. The therapist produces the reference example and the client attempts to produce the same example correctly. The example can be a syllable, a word, a pair or a sentence according to the client’s level, wishes and needs.
Gradually
the
clients learn how
to interpret their spreadlines or the dots in the cochleagrams and can
easily
compare their productions with the model one. But until that happens or
for
very young children who cannot readily compare the two screens, Box of
Tricks
gives another type of feedback which can be easily understood. This
automatic
feedback is placed under the cochleagrams and can take different forms.
Every
form has five stages in order to demonstrate any subtle improvement or
deterioration of each production. So the feedback can take the form of
a duck
which moves to the right and lifts its head in joy when the production
is
correct (Figure 6), a number which changes from 1 to 5 depending on the
production (Figure 7), a flower which comes gradually out of its pot as
the
production improves and a colour which changes from red to green when
the
production is correct. Figure 7 The automatic feedback can change through ‘Settings’ and take the form of a duck, a number, a flower or a colour. An additional feature of Box of Tricks which could be very helpful for the speech therapist is the ‘User Management’ tool. The therapist can have a login name and password and create files for all his or her clients. These files can be created during therapy by saving the client’s productions. Thus a database is created which contains the date of the recording, the type of exercise and the exact utterance which can be also played back (button ‘Say’ in Figure 8), the mark and the comments of the therapist at that time. Figure 8 This shows the selected files of a certain client. These files can be edited and can be confidential if the therapist chooses so. Thus a database is created and the therapist can easily keep track of the client’s progress. For more information about the project SPECO and the product Box of Tricks you can visit the official website: http://luna.ttt.bme.hu/speech/speco1.htm. If you wish to order the product you can get information about prices and other features of Box of Tricks from the company’s website: http://www.rcs.hu/sc.htm or email the company manager, Peter Barczikay: bpeter@rcs.hu. |
![Cochleagram of syllable [3:s]](fig5.jpg)
