Toward Accurate Assessment of Real-World Hearing Aid Benefit

The need for accurate, valid, and efficient tools for assessing the real-world benefits of directional hearing aids seems obvious. The US Food and Drug Administration (FDA) has called for multiple clinical trials that substantiate advertising claims made by hearing aid manufacturers. But field studies—where subjects go out into the world and report their personal observations—can be time consuming and costly, and have the uncertainties inherent in self-reporting.

A laboratory measurement system that reflects real-world results could make a significant contribution to hearing aid research and development. But, even when the potential benefits of a given instrument are known for the average user, the benefits for individual users may vary widely. Therefore, a clinical tool that predicts the real-world performance of a particular hearing aid for an individual patient would also be important.

This article discusses problems associated with the measurement and reporting of directional hearing-aid benefit under simulated real-world noise conditions, and describes the development of a laboratory sound system that is designed to solve many of those problems.

Traditional Measurements Can Give Misleading Results
In the history of testing speech intelligibility in noise, the number of protocols used has been almost as great as the number of studies. Consequently, one cannot readily compare results from one study to the next.

More importantly, most of the protocols reported in research to date yield results that are not likely to be applicable to real-world conditions. As an illustration, Figure 1 shows two traditional setups for testing the performance of directional microphone hearing aids in noise. A speech intelligibility test is delivered from a front loudspeaker while competing noise is delivered from a second loudspeaker placed either to the side or to the rear. In a third common arrangement, competing noise is delivered from an overhead loudspeaker. Any one of these setups can work well for testing the real-world performance of hearing aids that use omnidirectional microphones, but with directional microphones these setups fall short.

In the examples of Figure 1, the adjoining circles extending forward and behind the listener represent the pickup pattern of a bidirectional microphone. Bidirectional microphones are not usually found in hearing aids, but they have the same directivity index (DI) as cardioid microphones (commonly found in directional hearing aids) and will serve as an example here. Bidirectional microphones pick up sounds arriving from the front or the rear while rejecting sounds arriving from the sides or overhead.

In the example in Figure 1a, the only loudspeaker radiating noise points directly at the rejection null of the bidirectional pattern. In the real world, some of the competing noise (ordinarily from all directions) would not be attenuated by the null of the microphone. Therefore, the perceived improvement in signal-to-noise ratio (SNR) would be better in the test system than in the real world, yielding misleading results.

Alternately, in Figure 1b, the rear loudspeaker radiating noise points at a lobe of the bidirectional pattern. In this case, the SNR perceived by the listener in the real-world would be better than in the test setup. In real-world environments some of the competing noise would likely come from the sides, allowing the null of the directional pattern to offer an advantage not observed in this setup.

In short, because real-world environments present noise from many directions, dual-loudspeaker test systems cannot adequately represent real-world performance with directional microphones. A multiple-loudspeaker array is needed to reveal accurate results in testing the real-world performance of hearing aids that offer directional pickup patterns. But how many loudspeakers are necessary, and where should they be placed?

Multiple-Loudspeaker Test Setups and Their Limitations
Gerzon¹ showed that a first-order directional pattern (cardioid, bidirectional, or anything in between) must be sampled at least twice in a half circumference—four loudspeakers in the full circumference—for an accurate representation of the horizontal pickup pattern. In other words, to test the horizontal response of a first-order directional microphone in a free field, a horizontal array of four equally spaced loudspeakers does as good a job as a horizontal ring of an infinite number of loudspeakers. A second-order directional microphone requires twice as many sample points (eight loudspeakers) for the same accuracy, and so on.

So does this mean that an accurate sound-field simulator would need only four loudspeakers for testing the first-order directional microphones commonly found in today’s hearing aids? The answer is no. When a first-order microphone in a hearing aid is worn on a head, what starts out as a simple, first-order directional pattern can take on a complex set of higher-order patterns that vary with frequency—thus requiring a greater number of loudspeakers for sampling accuracy.

As an example, Figure 2 shows the horizontal polar patterns of an Etymotic Research CC-MIC, as worn in a mini-canal shell on KEMAR, for three frequencies. When measured in a free field, the horizontal polar pattern of the CC-MIC is nearly bidirectional (as in Figure 1), but when worn in a mini-canal shell the polar pattern takes on its intended, although more complex, characteristic. Further, other new hearing aids are entering the market that have free-field patterns of higher than first-order.² Even a four-loudspeaker array, therefore, has a potential for error when used to assess the SNR benefit of head-worn directional hearing aids.

Figures 3 and 4 illustrate the degree of possible errors in assessing SNR benefit when interfering noise is presented from various loudspeaker arrangements. The data were obtained using an electroacoustics computer program developed by the third author (Julstrom). The program calculates with a high degree of precision the outputs of complex directional patterns under various conditions. In Figures 3 and 4, the “SNR Error” is the SNR observed using a given loudspeaker arrangement as compared to what would be observed using an infinite number of loudspeakers in a horizontal ring. The target signal comes from a front-center loudspeaker, and interfering noise comes from the loudspeaker arrangement shown, assuming equal noise energy radiated from each noise loudspeaker. The total noise energy at the center of the array is the same for each example.

Figure 3 provides examples of the degree of error possible when assessing the improvement in SNR afforded by a 3-element array microphone³ worn on a head, assuming the potential for ±15° of head rotation during testing. The ranges of errors at 4 kHz and 2 kHz using 8 equally spaced noise loudspeakers are substantially less than those for 4 equally spaced noise loudspeakers in a corner arrangement, when minor head movements are accounted for.

Some commonly used loudspeaker arrangements have even greater potential for error in assessing real-world benefit. Figure 4 gives an example of a test setup that presents no noise from the front; only the target signal comes from the front of the listener. This setup implies that users of directional hearing aids will somehow be able to arrange their listening environments so that all the competing noise is to the sides and rear and only the target signal comes from the front. In real-world environments, even when no competing noises originate directly from the front, reflections of noises from other directions still provide significant noise energy from the front, where directional microphones have their greatest sensitivity. The graph in Figure 4 indicates the substantial overestimate of SNR improvement that can result from the 90º/180º/270º arrangement of noise loudspeakers, with the signal coming from the front speaker, and with the given microphones in a free-field condition. Adding a front noise loudspeaker essentially eliminates the observed error (within 0.6 dB).

In summary, even test setups that use multiple loudspeakers can lead to errors in assessing the real-world performance of hearing aids that use directional microphones. These errors are associated with two factors:

1. An insufficient number of loudspeakers, leading to excessive variation in noise response with minor head rotations, and
2. The absence of noise from the front, where directional microphones are the most sensitive.

While an array of 4 equally spaced noise loudspeakers can produce reasonably low errors for many directional devices under free-field conditions, our calculated data indicate that an array of 8 equally spaced noise loudspeakers can significantly reduce SNR errors under head-worn conditions that allow for minor head rotations.

Other Considerations for a Real-World Assessment Tool
So far, this article has focused on potential errors in assessing the directional microphone benefit using various loudspeaker arrangements to simulate real-world conditions. But there are other, more subjective factors that can affect accuracy in a sound-field simulation.

One of these is the fact that in most real-world listening conditions, the level and the spectrum of interfering noise fluctuate from moment to moment. Therefore, tests of hearing in noise that use stationary interfering noises (such as speech-shaped noise) lack a potentially important degree of face validity relative to real-world, fluctuating noise conditions.

A second characteristic of real-world environments that can affect perception involves the correlation of sound sources. For any symmetrical signal configuration surrounding a listener, if the signals from all sources are correlated (that is, if they have the same waveform), then the sound will be perceived as coming from the center of the head. Most of us have experienced this phenomenon when listening to recordings under headphones. In contrast, when noise sources are uncorrelated they appear distinct and separated. For example, if noises presented by an array of 8 surrounding loudspeakers were completely uncorrelated, the listener would perceive 8 distinct sources corresponding to the 8 discrete locations of the loudspeakers. Allowing a degree of controlled correlation between the 8 sound sources provides for a smoother, more seamless subjective rendering of a continuous 360° band of horizontal sounds.

Developing a System that Reproduces Real-World Sound
Figure 5 shows the loudspeaker arrangement for the research version of the R-SPACE™ sound-field reproduction and simulation system. The word “reproduction” is used, in addition to “simulation,” because the R-SPACE system is designed to reproduce an actual environment from a recording, rather than to build an artificial simulation from arbitrary elements (which this system can also do).

An array of 8 loudspeakers surrounds the listener. Using a patent-pending apparatus and method, a multi-channel audio recording with 8 discrete, yet partially correlated, signals recreates a life-like acoustic environment at the listening position in the center of the array.

The development of this system focused on two general goals:

1. Subjectively, the reproduced environments should “sound like” the real environments they reproduce;
2. Objectively, the reproduced environments should allow hearing aids using directional and omnidirectional microphones, as well as the open ear, to perform as they do in the real world.

To meet these goals, specific design and validation criteria were formulated. The first was that the system should provide subjectively accurate spatial, spectral, and dynamic qualities with respect to the real-world environments being reproduced. The first tests, for spatial qualities, were informal “walk-around” tests. One of the co-authors would walk around the microphone setup during recording while talking and making various noises (like crumpling a cellophane wrapper). Then we would listen to the playback of this recording to judge the spatial accuracy of the reproduction—in other words, to be certain that sound images were well focused and appeared to come from the correct locations. After trying several different recording and playback designs, we became satisfied that the system adequately represented life-like spatial qualities. All judges accurately reported the location of reproduced sound sources, and reported that they could close their eyes and “hear” the other person walking around them in a realistic fashion.

The second test was that the reproduced environments should be judged by ordinary listeners to sound real. We recorded a “dessert party” in the lunchroom at Etymotic Research using the patent-pending multi-track recording method, plus simultaneous binaural recordings using a KEMAR manikin fitted with ITE hearing-aid microphones. Ten subjects who had attended the original dessert party listened over Etymotic Research ER-4B insert earphones to the “live” binaural recordings, as well as to binaural ITE recordings made subsequently in the R-SPACE reproduction. Compton3 conducted the listening tests with the attendees, and found that her subjects judged the binaural recordings made in the R-SPACE system to be nearly as real-sounding as the recordings made at the actual party.

The third design criterion was that the simulation should objectively and accurately reveal the real-world benefits of hearing aids designed to improve hearing in noise. Because a noisy restaurant is one of the most difficult environments for hearing-impaired listeners, we made a new set of multi-track recordings at Lou Malnati’s, a noisy restaurant near the Etymotic Research laboratory. Again we made simultaneous KEMAR binaural recordings—this time using three sets of directional microphones having a wide range of directivity. These binaural recordings served as the standard “live” conditions. The multi-track recordings were then reproduced in the R-SPACE system, where additional KEMAR binaural recordings were made using the same directional microphones as in the restaurant. Subjects, listening to the binaural recordings with insert earphones, completed a series of clinical hearing-in-noise tests under “live” and “R-SPACE” conditions.

This clinical validation study was the subject of a doctoral dissertation by Compton.⁴ The results, in brief, showed that speech-intelligibility-in-noise scores (via the HINT test) in the R-SPACE conditions were statistically indistinguishable from the scores in the real restaurant conditions for omni ITE, first-order directional ITE, and 5-element ultradirectional array microphones. All microphones were worn as binaural, head-worn pairs on the KEMAR manikin. Thus, we concluded that the R-SPACE sound-field system met the goal of accurately revealing real-world benefit in a broad range of directional hearing aid microphones in a noisy restaurant.

With this system, it is now practical, for example, to generate exactly reproducible KEMAR recordings of an unlimited number of hearing aids that will provide intelligibility comparisons nearly identical to those made in the original real-world environment. Thus, the authors believe that a major problem in hearing aid research has been solved: the previously almost impossible task of comparing various hearing aids under realistic real-world conditions.2

Variability of SNRs in Real-World Test Materials
Aside from clinical validation, efficiency of use is another concern. A factor that can greatly affect the efficiency of such test systems in both research and clinical settings is the across-item equivalency of test materials. When lists of sentences are used to test speech intelligibility in noise, efficiency in obtaining reliable scores requires that the probability of meeting a criterion level of performance (e.g., 50% correct) should be the same for each test item and each test list. Without such across-item equivalency, obtaining scores that have statistical reliability would be far more time-consuming and/or expensive.

Existing clinical test materials that may have been validated for equivalency under a given set of test conditions (eg, IEEE sentences against 4-talker babble, as in the QuickSIN test) must be validated anew for any new set of test conditions—such as when presented with a reproduction of real restaurant noise using multiple loudspeakers. One sentence may have higher intelligibility than another in a given noise, and the noise level in a restaurant is constantly varying: A given sentence coupled with a different noise may have several dB lower or higher effective SNR.

Etymotic Research and Revitronix are currently engaged in clinical validation studies for across-item equivalency of speech-intelligibility materials in the R-SPACE restaurant reproduction, using: 1) The Bamford-Kowal-Bench (BKB) sentences⁵ recorded by Jon Shallop for Cochlear Americas, and 2) The spectrally corrected version of the Massachusetts Institute of Technology recordings of the IEEE sentences⁶ originally used with the SIN Test.

Toward a New Standard
Many workers within the hearing research community have expressed a need for standardization of methods used in the assessment and reporting of real-world benefits for new hearing aid designs.⁷ The authors believe the R-SPACE system provides a viable answer to this need. The currently available R-SPACE system was designed primarily for research applications. A streamlined, clinical version is under development, with completion projected for next year.

Acknowledgments
The R-SPACE sound-field system was jointly developed by Etymotic Research Inc, Elk Grove Village, Ill, and Revitronix, Brownsville, Vt. The polar data used to produce Figures 2 and 3 were measured by Diane Green. Special appreciation is given to Mead Killion and Edgar Villchur for their helpful and important comments on an earlier version of this manuscript.

References
1. Gerzon M. Calculating the directivity factor g of transducers from limited polar diagram information. J Audio Eng Soc. 1975; 23(5):369-373.
2. Christensen LA, Helmink D, Soede W, Killion MC. Complaints about hearing in noise: A new answer. Hearing Review. 2002;9(6):34-36.
3. Soede W, Bilsen FA, Berkhout AJ. Development of a directional hearing instrument based on array technology. J Acoust Soc Amer. 1990;94(2):799-808.
4. Compton CL. The validation of a sound field simulator for measurement of the real world performance of directional microphones for hearing aids. Dissertation Abstracts International. 2002. In press.
5. Bench J, Kowal A, Bamford J. BKB (Bamford-Kowal-Bench) sentence lists for partially-hearing children. Brit Jour of Audiol. 1979;13:108-112.
6. Fikret-Pasa S. The effects of compression ratio on speech intelligibility and quality [dissertation]. Northwestern University;1993.
7. Valente M. Personal communication, 2002.