Microphone use

This page provides a quick overview of how to access the microphones on Navel.

Devices

Navel’s 7 microphones feed ODAS for beamforming and sound-source tracking. The default capture chain feeds the ODAS output into the echo cancellation (EC) service before making processed audio available to applications:

Microphones -> ODAS -> EC (WebRTC AEC3) -> Application
                       ^
                       |
                Playback loopback

EC uses WebRTC’s AEC3 acoustic echo canceller, with a hardware loopback of Navel’s speaker playback as its reference. It suppresses the robot’s own playback in ODAS’ output.

Applications should use the ALSA default capture device for this processed audio. The supplied configuration in /etc/alsa/conf.d/70-ec.conf routes default capture to eco_1, which can also be selected explicitly. It provides mono audio in signed 16-bit little-endian PCM (S16_LE) format at 48 kHz.

Note

The ODAS capture devices are currently unavailable for application capture in the default chain. EC consumes the samples from ODAS’s FIFO (named pipe), so another reader would compete for those samples instead of receiving a separate copy. Use the default capture device to access the processed audio.

EC capture also uses a shared FIFO. Open one capture stream at a time and distribute its samples within your application if needed.

Speech recognition

Use the default capture device for speech recognition to receive audio after echo cancellation. The included /usr/share/navel/examples/chat/chat.py example uses this default microphone.

Sound source tracking

ODAS still provides sound-source tracking metadata independently of audio capture. It remains available while EC consumes the ODAS audio. Call the next_frame() method of Robot to receive a PerceptionData frame. Its sst_tracks_latest list contains SstMeta objects with each track’s ID, location, activity level, and is_dynamic flag.

Tracking happens before echo cancellation, so this metadata describes the sources detected by ODAS. It can be used independently of recording audio.

Only the direction of sounds can be calculated, not their exact position in 3D space. The directions are mapped to the surface of a sphere with a radius of 2 m before being added to the perception data. The loc attribute of each SstMeta object therefore represents the perceived direction, not a measured distance or the exact location of the sound source.

The following example prints the reported direction of tracks with an activity level above 0.2:

 1import asyncio
 2
 3import navel
 4
 5
 6async def main():
 7    print("Listening forever, press Ctrl+C to stop...")
 8    async with navel.Robot() as robot:
 9        while True:
10            perc = await robot.next_frame()
11            for metadata in perc.sst_tracks_latest:
12                if metadata.activity > 0.2:
13                    print(f"ODAS track {metadata.id}: {metadata.loc}")
14
15
16if __name__ == "__main__":
17    asyncio.run(main())

Recording

To save audio to a file or pass it to another speech recognition framework, open the default capture device. The example below uses the PyAudio library, which exposes an API for audio streams. Install it in your Python environment with pip install pyaudio. This example records five seconds of processed audio into a mono WAV file at 48 kHz:

 1import wave
 2
 3import pyaudio
 4
 5SAMPLE_RATE = 48000
 6BYTES_PER_SAMPLE = 2
 7CHANNELS = 1
 8
 9
10def main():
11    device = "default"
12    rec_len = 5
13    path = "output.wav"
14
15    p = pyaudio.PyAudio()
16    try:
17        print(f"Recording {rec_len} seconds of audio from {device} to {path}")
18        stream = p.open(
19            format=pyaudio.paInt16,
20            channels=CHANNELS,
21            input_device_index=get_audio_device_index(p, device),
22            rate=SAMPLE_RATE,
23            input=True,
24        )
25        try:
26            with wave.open(path, "wb") as fp:
27                fp.setnchannels(CHANNELS)
28                fp.setsampwidth(BYTES_PER_SAMPLE)
29                fp.setframerate(SAMPLE_RATE)
30
31                frames_remaining = SAMPLE_RATE * rec_len
32                while frames_remaining:
33                    frames = min(SAMPLE_RATE // 10, frames_remaining)
34                    fp.writeframesraw(stream.read(frames))
35                    frames_remaining -= frames
36        finally:
37            stream.close()
38    finally:
39        p.terminate()
40
41
42def get_audio_device_index(p: pyaudio.PyAudio, device: str) -> int:
43    info = p.get_host_api_info_by_type(pyaudio.paALSA)
44
45    for i in range(info["deviceCount"]):
46        dev = p.get_device_info_by_host_api_device_index(info["index"], i)
47
48        if device == dev["name"] and dev["maxInputChannels"] > 0:
49            return dev["index"]
50
51    raise ValueError(f"ALSA capture device '{device}' does not exist")
52
53
54if __name__ == "__main__":
55    main()

Note

This example uses PyAudio’s blocking API. When recording alongside asynchronous robot control, run it in a worker thread (for example with asyncio.to_thread) or use PyAudio’s callback API so that audio reads do not block the event loop. Consult the PyAudio documentation for details.