lib_mic_array: PDM microphone array library£££doc/rst/lib_mic_array.html#lib-mic-array-pdm-microphone-array-library

lib_mic_array: PDM microphone array library$$$Overview$$$High-Level Process View$$$Execution Contexts£££doc/rst/src/overview.html#execution-contexts

The mic array unit uses two different execution contexts. The first is the PDM rx service (“PDM rx”), which is responsible for reading PDM samples from the physical port, and has relatively little work to do, but also has a strict real-time constraint on reading port data in a timely manner. The second is the decimation thread, which is where all processing other than PDM capture is performed.

This two-context model relaxes the need for tight coupling and synchronization between PDM rx and the decimation thread, allowing significant flexibility in how samples are processed in the decimation thread.

PDM rx is typically run within an interrupt on the same hardware core as the decimation thread, but it can also be run as a separate thread in cases where many channels result in a high processing load.

Likewise, the decimators may be split into multiple parallel hardware threads in the case where the processing load exceeds the MIPS available in a single thread.

lib_mic_array: PDM microphone array library$$$Overview$$$High-Level Process View$$$Step 1: PDM Capture£££doc/rst/src/overview.html#step-1-pdm-capture

The PDM data signal is captured by the xcore.ai device’s port hardware. The port receiving the PDM signals buffers the received samples. Each time the port buffer is filled, PDM rx reads the received samples.

Samples are collected word-by-word and assembled into blocks. Each time a block has been filled, the block is transferred to the decimation thread where all remaining mic array processing takes place.

The size of PDM data blocks varies depending upon the configured number of microphone channels and the configured second stage decimator’s decimation factor. Each PDM data block will contain exactly enough PDM samples to produce one new mic array (multi-channel) output sample.

lib_mic_array: PDM microphone array library$$$Overview$$$High-Level Process View$$$Step 2: First Stage Decimation£££doc/rst/src/overview.html#step-2-first-stage-decimation

The conversion from the high-sample-rate PDM stream to lower-sample-rate PCM stream involves two stages of decimating filters. After the decimation thread receives a block of PDM samples, the samples are filtered by the first stage decimator.

The first stage decimator has a fixed decimation factor of 32 and a fixed tap count of 256. An application is free to supply its own filter coefficients for the first stage decimator (using the fixed decimation factor and tap count), however this library also provides a reference filter for the first stage decimator that is recommended for most applications.

The first stage decimating filter is an FIR filter with 16-bit coefficients, and where each input sample corresponds to a +1 or a -1 (typical for PDM signals). The output of the first stage decimator is a block of 32-bit PCM samples with a sample time 32 times longer than the PDM sample time.

See Decimator Stages for further details.

lib_mic_array: PDM microphone array library$$$Overview$$$High-Level Process View$$$Step 3: Second Stage Decimation£££doc/rst/src/overview.html#step-3-second-stage-decimation

The second stage decimator is a decimating FIR filter with a configurable decimation factor and tap count. Like the first stage decimator, this library provides a reference filter suitable for the second stage decimator. The supplied filter has a tap count of 65 and a decimation factor of 6.

The output of the first stage decimator is a block of N*K PCM values, where N is the number of microphones and K is the second stage decimation factor. This is just enough samples to produce one output sample from the second stage decimator.

The resulting sample is vector-valued (one element per channel) and has a sample time corresponding to 32*K PDM clock periods. Using the reference filters and a 3.072 MHz PDM clock, the output sample rate is 16 kHz.

See Decimator Stages for further details.

lib_mic_array: PDM microphone array library$$$Overview$$$High-Level Process View$$$Step 4: Post-Processing£££doc/rst/src/overview.html#step-4-post-processing

After second stage decimation, the resulting sample goes to post-processing where two (optional) post-processing steps are available.

The first is a simple IIR filter, called DC Offset Elimination, which seeks to ensure each output channel tends to approach zero mean. DC Offset Elimination can be disabled if not desired. See Sample Filters for further details.

The second post-processing step is framing, where instead of signaling each sample of audio to subsequent processing stages one at a time, samples can be aggregated and transferred to subsequent processing stages as non-overlapping blocks. The size of each frame is configurable (down to 1 sample per frame, where framing is functionally disabled).

Finally, the sample or frame is transmitted over a channel from the mic array module to the next stage of the processing pipeline.

lib_mic_array: PDM microphone array library$$$Overview$$$High-Level Process View$$$Extending/Modifying Mic Array Behavior£££doc/rst/src/overview.html#extending-modifying-mic-array-behavior

At the core of lib_mic_array are several C++ class templates which are loosely coupled and intended to be easily overridden for modified behavior. The mic array unit itself is an object made by the composition of several smaller components which perform well-defined roles.

For example, modifying the mic array unit to use some mechanism other than a channel to move the audio frames out of the mic array is a matter of defining a small new class encapsulating just the modified transfer behavior, and then instantiating the mic array class template with the new class as the appropriate template parameter.

With that in mind, while most applications will have no need to modify the mic array behavior, it is nevertheless designed to be easy to do so.

lib_mic_array: PDM microphone array library$$$Getting Started$$$Identify Resources$$$Clock Blocks£££doc/rst/src/getting_started.html#clock-blocks

While clock blocks may be more abstract than ports, their implications for this library are actually simpler. First, the mic array unit will need a way of taking the audio master clock and dividing it to produce a PDM sample clock. This can be accomplished with a clock block. This will be the clock block which the API documentation refers to as “Clock A”.

Second, if (and only if) the PDM microphones are being used in a Dual Data Rate (DDR) configuration a second clock block will be required. In a DDR configuration 2 microphones share a physical pin for output sample data, where one signals on the rising edge of the PDM clock and the other signals on the falling edge. The second clock block required in a DDR configuration is referred to as “Clock B” in the API documentation.

Each tile on an xcore.ai device has 5 clock blocks available. In code, a clock block is identified by its resource ID, which are given as the preprocessor macros XS1_CLKBLK_1 through XS1_CLKBLK_5.

Unlike ports, which are tied to specific physical pins, clock blocks are fungible. Your application is free to use any clock block that has not already been allocated for another purpose. The vanilla component model defaults to using XS1_CLKBLK_1 and XS1_CLKBLK_2.

lib_mic_array: PDM microphone array library$$$Getting Started$$$Identify Resources$$$Ports£££doc/rst/src/getting_started.html#ports

Three ports are needed for the mic array component. As mentioned above, ports are physically tied to specific device pins, and so the correct ports must be identified for correct behavior.

Note that while ports are physically tied to specific pins, this is not a 1-to-1 mapping. Each port has a port width (measured in bits) which is the number of pins which comprise the port. Further, the pin mappings for different ports overlap, with a single pin potentially belonging to multiple ports. When identifying the needed ports, take care that both the pin map (see the documentation for your xcore.ai package) and port width are correct.

The first port needed is a 1-bit port on which the audio master clock is received. In the documentation, this is usually referred to as p_mclk.

The second port needed is a 1-bit port on which the PDM clock will be signaled to the PDM mics. This port is referred to as p_pdm_clk.

The third port is that on which the PDM data is received. In an SDR configuration, the width of this port must be greater than or equal to the number of microphones. In a DDR configuration, twice this port width must be greater than or equal to the number of microphones. This port is referred to as p_pdm_mics.

XCore applications are typically compiled with an “XN” file (with a “.xn” file extension). An XN file is an XML document which describes some information about the device package as well as some other helpful board-related information. The identification of your ports may have already been done for you in your XN file. Following is a snippet from an XN file with mappings for the three ports described above:

...
<Tile Number="1" Reference="tile[1]">
  <!-- MIC related ports -->
  <Port Location="XS1_PORT_1G"  Name="PORT_PDM_CLK"/>
  <Port Location="XS1_PORT_1F"  Name="PORT_PDM_DATA"/>
  <!-- Audio ports -->
  <Port Location="XS1_PORT_1D"  Name="PORT_MCLK_IN_OUT"/>
  <Port Location="XS1_PORT_1C"  Name="PORT_I2S_BCLK"/>
  <Port Location="XS1_PORT_1B"  Name="PORT_I2S_LRCLK"/>
  <!-- Used for looping back clocks -->
  <Port Location="XS1_PORT_1N"  Name="PORT_NOT_IN_PACKAGE_1"/>
</Tile>
...

The first 3 ports listed, PORT_PDM_CLK, PORT_PDM_DATA and PORT_MCLK_IN_OUT are respectively p_pdm_clk, p_pdm_mics and p_mclk. The value in the Location attribute (e.g. XS1_PORT_1G) is the port name as you will find it in your package documentation.

In this case, either PORT_PDM_CLK or XS1_PORT_1G can be used in code to identify this port.

lib_mic_array: PDM microphone array library$$$Getting Started$$$Identify Resources$$$Declaring Resources£££doc/rst/src/getting_started.html#declaring-resources

Once the ports and clock blocks to be used have been identified, these resources can be represented in code using a pdm_rx_resources_t struct. The following is an example of declaring resources in a DDR configuration. See pdm_rx_resources_t, PDM_RX_RESOURCES_SDR() and PDM_RX_RESOURCES_DDR() for more details.

pdm_rx_resources_t pdm_res = PDM_RX_RESOURCES_DDR(
                                PORT_MCLK_IN_OUT,
                                PORT_PDM_CLK,
                                PORT_PDM_DATA,
                                XS1_CLKBLK_1,
                                XS1_CLKBLK_2);

Note that this is not necessary in applications using the vanilla model.

lib_mic_array: PDM microphone array library$$$Getting Started$$$Identify Resources$$$Other Resources£££doc/rst/src/getting_started.html#other-resources

In addition to ports and clock blocks, there are also several other hardware resource types used by lib_mic_array which are worth considering. Running out of any of these will preclude the mic array from running correctly (if at all)

• Threads - At least one hardware thread is required to run the mic array component.

• Compute - The mic array unit will require a fixed number of MIPS (millions of instructions per second) to perform the required processing. The exact requirement will depend on the configuration used.

• Memory - The mic array requires a modest amount of memory for code and data. (see Mic Array Resource Usage).

• Chanends - At least 4 chanends must be available for signaling between threads/sub-components.

lib_mic_array: PDM microphone array library$$$Getting Started$$$Vanilla Model$$$Vanilla - CMake Macro£££doc/rst/src/getting_started.html#vanilla-cmake-macro

To simplify this further, a CMake macro called mic_array_vanilla_add() has been included with the build system.

mic_array_vanilla_add() takes several arguments:

• TARGET_NAME - The name of the CMake application target that the vanilla mode source should be added to.

• MCLK_FREQ - The frequency of the master audio clock, in Hz.

• PDM_FREQ - The desired frequency of the PDM clock, in Hz.

• MIC_COUNT - The number of microphone channels to be captured.

• SAMPLES_PER_FRAME - The size of the audio frames produced by the mic array unit (frames will be 2 dimensional arrays with shape (MIC_COUNT,SAMPLES_PER_FRAME)).

lib_mic_array: PDM microphone array library$$$Getting Started$$$Vanilla Model$$$Vanilla - Optional Configuration£££doc/rst/src/getting_started.html#vanilla-optional-configuration

Though not exposed by the mic_array_vanilla_add() macro, several additional configuration options are available when using the vanilla model. These are all configured by adding defines to the application target.

lib_mic_array: PDM microphone array library$$$Getting Started$$$Vanilla Model$$$Vanilla - Initializing and Starting£££doc/rst/src/getting_started.html#vanilla-initializing-and-starting

Once the configuration options have been chosen, initializing and starting the mic array at run-time is easily achieved. Two function calls are necessary, both are included through mic_array_vanilla.h (which was added to your include path through your build configuration).

First, during application initialization, the function ma_vanilla_init(), which takes no arguments, must be called. This will configure the hardware resources and install the PDM rx service as an ISR, but will not actually start any threads or PDM capture.

Once any remaining application initialization is complete, PDM capture and processing is started by calling ma_vanilla_task(). ma_vanilla_task() is a blocking call which takes a single argument which is the chanend that will be used to transmit audio frames to subsequent stages of the processing pipeline. Usually the call to ma_vanilla_task() will be placed directly in a par {...} block along with other threads to be started on the tile.

lib_mic_array: PDM microphone array library$$$Getting Started$$$Prefab Model$$$Prefab - Declare Resources£££doc/rst/src/getting_started.html#prefab-declare-resources

The example in this section will use 2 microphones in a DDR configuration with DC offset elimination enabled, and using 128-sample frames. The resource IDs used may differ than those required for your application.

pdm_res will be used to identify the ports and clocks which will be configured for PDM capture.

Within a C++ source file:

#include "mic_array/mic_array.h"
...
#define MIC_COUNT    2    // 2 mics
#define DCOE_ENABLE  true // DCOE on
#define FRAME_SIZE   128  // 128 samples per frame
...
pdm_rx_resources_t pdm_res = PDM_RX_RESOURCES_DDR(
                                PORT_MCLK_IN_OUT,
                                PORT_PDM_CLK,
                                PORT_PDM_DATA,
                                MIC_ARRAY_CLK1,
                                MIC_ARRAY_CLK2);
...

lib_mic_array: PDM microphone array library$$$Getting Started$$$Prefab Model$$$Prefab - Allocate MicArray£££doc/rst/src/getting_started.html#prefab-allocate-micarray

The C++ class template MicArray is central to the mic array unit in this library. The class templates defined in the mic_array::prefab namespace each derive from mic_array::MicArray.

Define and allocate the specific implementation of MicArray to be used.

...
// Using the full name of the class could become cumbersome. Using an alias.
using TMicArray = mic_array::prefab::BasicMicArray<
                      MIC_COUNT, FRAME_SIZE, DCOE_ENABLED>
// Allocate mic array
TMicArray mics = TMicArray();
...

Now the mic array unit has been defined and allocated. The template parameters supplied (e.g. MIC_COUNT and FRAME_SIZE) are used to calculate the size of any data buffers required by the mic array, and so the mics object is self-contained, with all required buffers being statically allocated. Additionally, class templates will ultimately allow unused features to be optimized out at build time. For example, if DCOE is disabled, it will be optimized out at build time so that at run time it won’t even need to check whether DCOE is enabled.

lib_mic_array: PDM microphone array library$$$Getting Started$$$Prefab Model$$$Prefab - Init and Start Functions£££doc/rst/src/getting_started.html#prefab-init-and-start-functions

Now a couple functions need to be implemented in your C++ file. In most cases these functions will need to be callable from C or XC, and so they should not be static, and they should be decorated with extern "C" (or the MA_C_API preprocessor macro provided by the library).

First, a function which initializes the MicArray object and configures the port and clock block resources. The documentation for BasicMicArray indicates any parts of the MicArray object that need to be initialized.

#define MCLK_FREQ   24576000
#define PDM_FREQ    3072000
...
MA_C_API
void app_init() {
  // Configure clocks and ports
  const unsigned mclk_div = mic_array_mclk_divider(MCLK_FREQ, PDM_FREQ);
  mic_array_resources_configure(&pdm_res, mclk_div);

  // Initialize the PDM rx service
  mics.PdmRx.Init(pdm_res.p_pdm_mics);
}
...

app_init() can be called from an XC main() during initialization.

Assuming the PDM rx service is to be run as an ISR, a second function is used to actually start the mic array unit. This starts the PDM clock, install the ISR and enter the decimator thread’s main loop.

MA_C_API
void app_mic_array_task(chanend_t c_audio_frames) {
  mics.SetOutputChannel(c_audio_frames);

  // Start the PDM clock
  mic_array_pdm_clock_start(&pdm_res);

  mics.InstallPdmRxISR();
  mics.UnmaskPdmRxISR();

  mics.ThreadEntry();
}

Now a call to app_mic_array_task() with the channel to send frames on can be placed inside a par {...} block to spawn the thread.

lib_mic_array: PDM microphone array library$$$Decimator Stages$$$Decimator Stage 1$$$Filter Implementation (Stage 1)£££doc/rst/src/decimator_stages.html#filter-implementation-stage-1

The input to the first stage decimator (here called “Stream A”) is a stream of 1-bit PDM samples with a sample rate of PDM_FREQ. Rather than each PDM sample representing a value of 0 or 1, each PDM sample represents a value of either +1 or -1. Specifically, on-chip and in-memory, a bit value of 0 represents +1 and a bit value of 1 represents -1.

The output from the first stage decimator, Stream B, is a stream of 32-bit PCM samples with a sample rate of PDM_FREQ/S1_DEC_FACTOR = PDM_FREQ/32. For example, if PDM_FREQ is 3.072 MHz, then Stream B’s sample rate is 96.0 kHz.

The first stage filter is structured to make optimal use of the XCore XS3 vector processing unit (VPU), which can compute the dot product of a pair of 256-element 1-bit vectors in a single cycle. The first stage uses 256 16-bit coefficients for its filter taps.

The signature of the filter function is

int32_t fir_1x16_bit(uint32_t signal[8], uint32_t coeff_1[]);

Each time 32 PDM samples (1 word) become available for an audio channel, those samples are shifted into the 8-word (256-bit) filter state, and a call to fir_1x16_bit results in 1 Stream B sample element for that channel.

The actual implementation for the first stage filter can be found in src/fir_1x16_bit.S. Additional usage details can be found in api/etc/fir_1x16_bit.h.

Note that the 256 16-bit filter coefficients are not stored in memory as a standard coefficient array (i.e. int16_t filter[256] = {b[0], b[1], ... };). Rather, in order to take advantage of the VPU, the coefficients must be rearranged bit-by-bit into a block form suitable for VPU processing. See the section below on filter conversion if supplying a custom filter for stage 1.

lib_mic_array: PDM microphone array library$$$Decimator Stages$$$Decimator Stage 1$$$Provided Filter (Stage 1)£££doc/rst/src/decimator_stages.html#provided-filter-stage-1

This library provides filter coefficients that may be used with the first stage decimator. These coefficients are available in your application through the header mic_array/etc/filters_default.h as stage1_coef.

lib_mic_array: PDM microphone array library$$$Decimator Stages$$$Decimator Stage 1$$$Provided Filter (Stage 1)$$$Filter Characteristics (Stage 1)£££doc/rst/src/decimator_stages.html#filter-characteristics-stage-1

The plot below indicates the frequency response of the provided first stage decimation filter First stage decimation filter freq response.

lib_mic_array: PDM microphone array library$$$Decimator Stages$$$Decimator Stage 1$$$Filter Conversion Script£££doc/rst/src/decimator_stages.html#filter-conversion-script

Taking a set of floating-point coefficients, quantizing them into 16-bit coefficients and ‘boggling’ them into the correct memory layout can be a tricky business. To simplify this process, this library provides a Python (3) script which does this process for you.

The script can be found in this repository at python/stage1.py.

lib_mic_array: PDM microphone array library$$$Decimator Stages$$$Decimator Stage 2$$$Filter Implementation (Stage 2)£££doc/rst/src/decimator_stages.html#filter-implementation-stage-2

The input to the second stage decimator (here called “Stream B”) is the stream of 32-bit PCM samples emitted from the first stage decimator with a sample rate of PDM_FREQ/32.

The output from the second stage decimator, Stream C, is a stream of 32-bit PCM samples with a sample rate of PDM_FREQ/(32*S2_DEC_FACTOR). For example, if PDM_FREQ is 3.072 MHz, and S2_DEC_FACTOR is 6, then Stream C’s sample rate (the sample rate received by the main application code) is

3.072 MHz / (32*6) = 16 kHz

The second stage filter uses the 32-bit FIR filter implementation from lib_xcore_math. See xs3_filter_fir_s32() in that library for more implementation details.

lib_mic_array: PDM microphone array library$$$Decimator Stages$$$Decimator Stage 2$$$Provided Filter (Stage 2)£££doc/rst/src/decimator_stages.html#provided-filter-stage-2

This library provides a filter suitable for the second stage decimator. It is available in your application through the header mic_array/etc/filters_default.h.

For the provided filter S2_TAP_COUNT = 65, and S2_DEC_FACTOR = 6.

lib_mic_array: PDM microphone array library$$$Decimator Stages$$$Decimator Stage 2$$$Provided Filter (Stage 2)$$$Filter Characteristics (Stage 2)£££doc/rst/src/decimator_stages.html#filter-characteristics-stage-2

The plot below indicates the frequency response of the provided second stage decimation filter Second stage decimation filter freq response.

lib_mic_array: PDM microphone array library$$$Decimator Stages$$$Custom Filters$$$Configuring for 32 kHz or 48 kHz output£££doc/rst/src/decimator_stages.html#configuring-for-32-khz-or-48-khz-output

Filter design scripts are provided to support higher output sampling rates than the default 16 kHz.

Both stage 1 and stage 2 need to be updated because the first stage needs a higher cut off frequency before samples are passed to the downsample by three (32 kHz) or two (48 kHz) second stage decimator.

From the command line, follow these instructions:

python filter_design/design_filter.py # generate the filter .pkl files
python stage1.py good_32k_filter_int.pkl # convert the .pkl file to a C style array for stage 1
python stage2.py good_32k_filter_int.pkl # convert the .pkl file to a C style array for stage 2

Next copy the output from last two scripts into a source file. This could be your mic_array.cpp file which launches the mic array tasks. It may look something like this:

#define MIC_ARRAY_32K_STAGE_1_TAP_COUNT 148
#define MIC_ARRAY_32K_STAGE_1_FILTER_WORD_COUNT 128
static const uint32_t WORD_ALIGNED stage1_32k_coefs[MIC_ARRAY_32K_STAGE_1_FILTER_WORD_COUNT]
{
    .... the coeffs
};

#define MIC_ARRAY_32K_STAGE_2_TAP_COUNT 96
static constexpr right_shift_t stage2_32k_shift = 3;

static const int32_t WORD_ALIGNED stage2_32k_coefs[MIC_ARRAY_32K_STAGE_2_TAP_COUNT] = {
    .... the coeffs
};

The new decimation object must now be declared that references your new filter coefficients. Again, this example is for 32 kHz output since the decimation factor is 3.:

using TMicArray = mic_array::MicArray<mic_count,
    mic_array::TwoStageDecimator<mic_count,
                               3,
                               MIC_ARRAY_32K_STAGE_2_TAP_COUNT>,
    mic_array::StandardPdmRxService<MIC_ARRAY_CONFIG_MIC_IN_COUNT,
                                mic_count,
                                3>,
    typename std::conditional<MIC_ARRAY_CONFIG_USE_DC_ELIMINATION,
                                mic_array::DcoeSampleFilter<mic_count>,
                                mic_array::NopSampleFilter<mic_count>>::type,
    mic_array::FrameOutputHandler<mic_count,
                                MIC_ARRAY_CONFIG_SAMPLES_PER_FRAME,
                                mic_array::ChannelFrameTransmitter>>;

Next you need to change how you initialise and run the mic array task to reference your new mic array custom object. Normally the following code would be used in ma_init():

mics.Init();
mics.SetPort(pdm_res.p_pdm_mics);
mic_array_resources_configure(&pdm_res, MIC_ARRAY_CONFIG_MCLK_DIVIDER);
mic_array_pdm_clock_start(&pdm_res);

however if you wish to use custom filters then the initialisation would look like this:

mics.Decimator.Init(stage1_32k_coefs, stage2_32k_coefs, stage2_32k_shift);
mics.PdmRx.Init(pdm_res.p_pdm_mics);
mic_array_resources_configure(&pdm_res, MIC_ARRAY_CONFIG_MCLK_DIVIDER);
mic_array_pdm_clock_start(&pdm_res);

Finally, the ma_task() function needs to be changed from the default way of calling:

mics.SetOutputChannel(c_frames_out);
mics.InstallPdmRxISR();
mics.UnmaskPdmRxISR();
mics.ThreadEntry();

to using the custom version of the object:

mics.OutputHandler.FrameTx.SetChannel(c_frames_out);
mics.PdmRx.InstallISR();
mics.PdmRx.UnmaskISR();
mics.ThreadEntry();

The increased sample rate will place a higher MIPS burden on the processor. The typical MIPS usage (see section Mic Array Resource Usage) is in the order of 11 MIPS per channel using a 16 kHz output decimator.

Increasing the output sample rate to 32 kHz using the same length filters will increase processor usage per channel to approximately 13 MIPS rising to 15.6 MIPS for 48 kHz.

Increasing the filer lengths to 148 and 96 for stages 1 and 2 respectively at 48 kHz will increase processor usage per channel to around 20 MIPS.

lib_mic_array: PDM microphone array library$$$Decimator Stages$$$Custom Filters$$$Configuring for 32 kHz or 48 kHz output$$$Filter Characteristics for good_32k_filter_int.pkl£££doc/rst/src/decimator_stages.html#filter-characteristics-for-good-32k-filter-int-pkl

The plot below indicates the frequency response of the first and second stages of the provided 32 kHz filters as well as the cascaded overall response. Note that the overall combined response provides a nice flat passband as shown in the good_32k_filter_int.pkl frequency response.

lib_mic_array: PDM microphone array library$$$Decimator Stages$$$Custom Filters$$$Configuring for 32 kHz or 48 kHz output$$$Filter Characteristics for good_48k_filter_int.pkl£££doc/rst/src/decimator_stages.html#filter-characteristics-for-good-48k-filter-int-pkl

The plot below indicates the frequency response of the first and second stages of the provided 48 kHz filters as well as the cascaded overall response. Note that the overall combined response provides a nice flat passband as shown good_48k_filter_int.pkl frequency response.

lib_mic_array: PDM microphone array library$$$Sample Filters$$$DC Offset Elimination$$$Enabling/Disabling DCOE£££doc/rst/src/sample_filters.html#enabling-disabling-dcoe

Whether the DCOE filter is enabled by default and how to enable or disable it depends on which approach your project uses to include the mic array component in the application.

# Gather sources and create application target
# ...
# Add vanilla source to application build
mic_array_vanilla_add(my_app  ${MCLK_FREQ} ${PDM_FREQ}
                            ${MIC_COUNT} ${FRAME_SIZE} )
# ...
# Disable DCOE
target_compile_definitions(my_app
    PRIVATE MIC_ARRAY_CONFIG_USE_DC_ELIMINATION=0 )

template <unsigned MIC_COUNT, unsigned FRAME_SIZE, bool USE_DCOE>
    class BasicMicArray : public ...

#include "mic_array/mic_array.h"
...
// Controls whether DCOE is enabled
static constexpr bool enable_dcoe = true;
auto mics = mic_array::prefab::BasicMicArray<MICS, FRAME_SIZE, enable_dcoe>();
...

#include "mic_array/cpp/MicArray.hpp"
using namespace mic_array;
...
template <unsigned MIC_COUNT, class TDecimator,
          class TPdmRx, class TOutputHandler>
class DcoeEnabledMicArray : public MicArray<MIC_COUNT, TDecimator, TPdmRx,
                                    DcoeSampleFilter, TOutputHandler>
{
  ...
};

lib_mic_array: PDM microphone array library$$$Sample Filters$$$DC Offset Elimination$$$DCOE Filter Equation£££doc/rst/src/sample_filters.html#dcoe-filter-equation

As mentioned above, the DCOE filter is a simple IIR filter given by the following equation, where x[t] and x[t-1] are the current and previous input sample values respectively, and y[t] and y[t-1] are the current and previous output sample values respectively.

R = 252.0 / 256.0
y[t] = R * y[t-1] + x[t] - x[t-1]

lib_mic_array: PDM microphone array library$$$Sample Filters$$$DC Offset Elimination$$$DCOE Filter Frequency Response£££doc/rst/src/sample_filters.html#dcoe-filter-frequency-response

The plot below indicates the frequency response of DCOE filter DCOE filter frequency response.

lib_mic_array: PDM microphone array library$$$Software Structure$$$High-Level View$$$Mic Array / Decimator Thread£££doc/rst/src/software_structure.html#mic-array-decimator-thread

Aside from aggregating its sub-components into a single logical entity, the MicArray class template also holds the high-level logic for capturing, processing and coordinating movement of the audio stream data.

The following code snippet is the implementation for the main mic array thread (or “decimation thread”; not to be confused with (optional) PDM capture thread).

void mic_array::MicArray<MIC_COUNT,TDecimator,TPdmRx,
                                  TSampleFilter,
                                  TOutputHandler>::ThreadEntry()
{
  int32_t sample_out[MIC_COUNT] = {0};

  while(1){
    uint32_t* pdm_samples = PdmRx.GetPdmBlock();
    Decimator.ProcessBlock(sample_out, pdm_samples);
    SampleFilter.Filter(sample_out);
    OutputHandler.OutputSample(sample_out);
  }
}

The thread loops forever, and on each iteration

• Requests a block of PDM sample data from the PDM rx service. This is a blocking call which only returns once a complete block becomes available.

• Passes the block of PDM sample data to the decimator to produce a single output sample.

• Applies a post-processing filter to the sample data.

• Passes the processed sample to the output handler to be transferred to the next stage of the processing pipeline. This may also be a blocking call, only returning once the data has been transferred.

Note that the MicArray object doesn’t care how these steps are actually implemented. For example, one output handler implementation may send samples one at a time over a channel. Another output handler implementation may collect samples into frames, and use a FreeRTOS queue to transfer the data to another thread.

lib_mic_array: PDM microphone array library$$$Software Structure$$$High-Level View$$$Curiously Recurring Template Pattern£££doc/rst/src/software_structure.html#curiously-recurring-template-pattern

The C++ API of this library makes heavy use of the Curiously Recurring Template Pattern (CRTP).

Instead of providing flexibility through abstract classes or polymorphism, CRTP achieves flexibility through the use of class templates with type template parameters. As with derived classes and virtual methods, the CRTP template parameter must follow a contract with the class template where it implements one or more methods with specific names and signatures that the class template directly calls.

There are a couple notable advantages of using CRTP over polymorphic behavior. With CRTP flexibility does not generally come with the same run-time costs (in terms of both compute and memory) as polymorphic solutions. This is because the CRTP class template always knows the concrete type of any objects it uses at compile time. This avoids the need for run time type information or virtual function tables. This allows compile time optimizations can be made which may not be otherwise available. This in-turn allows many function calls to be inlined, or in some cases, entirely eliminated.

Additionally, while not strictly an example of CRTP, integer template parameters are also heavily used in class templates. The two main advantages of this are that it allows objects to encapsulate their own (statically allocated) memory, and that it allows the compiler to make compile time loop optimizations that it may not otherwise be able to make.

The downside to CRTP is that it tends to lead to highly verbose class type names, where templated classes end up with type parameter assignments are themselves templated classes with their own template parameters.

lib_mic_array: PDM microphone array library$$$Software Structure$$$High-Level View$$$Sub-Component Initialization£££doc/rst/src/software_structure.html#sub-component-initialization

Each of MicArray’s sub-components may have implementation-specific configuration or initialization requirements. Each sub-component is a public member of MicArray (see table above). An application can access a sub-component directly to perform any type-specific initialization or other manipulation.

For example, the ChannelFrameTransmitter output handler class needs to know the chanend to be used for sending samples. This can be initialized on a MicArray object mics with mics.OutputHandler.SetChannel(c_sample_out).

lib_mic_array: PDM microphone array library$$$Software Structure$$$Sub-Components$$$PdmRx£££doc/rst/src/software_structure.html#pdmrx

PdmRx, or the PDM rx service is the MicArray sub-component responsible for capturing PDM sample data, assembling it into blocks, and passing it along so that it can be decimated.

The MicArray class requires only that PdmRx implement GetPdmBlock(), a blocking call that returns a pointer to a block of PDM data which is ready for further processing.

Generally speaking, PdmRx will derive from the PdmRxService class template. PdmRxService encapsulates the logic of using an xCore port for capturing PDM samples one word (32 bits) at a time, and managing two buffers where blocks of samples are collected. It also simplifies the logic of running PDM rx as either an interrupt or as a stand-alone thread.

PdmRxService has 2 template parameters. The first is the BLOCK_SIZE, which specifies the size of a PDM sample block (in words). The second, SubType, is the type of the sub-class being derived from PdmRxService. This is the CRTP (Curiously Recurring Template Pattern), which allows a base class to use polymorphic-like behaviors while ensuring that all types are known at compile-time, avoiding the drawbacks of using virtual functions.

There is currently one class template which derives from PdmRxService, called StandardPdmRxService. StandardPdmRxService uses a streaming channel to transfer PDM blocks to the decimator. It also provides methods for installing an optimized ISR for PDM capture.

lib_mic_array: PDM microphone array library$$$Software Structure$$$Sub-Components$$$Decimator£££doc/rst/src/software_structure.html#decimator

The Decimator sub-component encapsulates the logic of converting blocks of PDM samples into PCM samples. The TwoStageDecimator class is a decimator implementation that uses a pair of decimating FIR filters to accomplish this.

The first stage has a fixed tap count of 256 and a fixed decimation factor of 32. The second stage has a configurable tap count and decimation factor.

For more details, see Decimator Stages.

lib_mic_array: PDM microphone array library$$$Software Structure$$$Sub-Components$$$SampleFilter£££doc/rst/src/software_structure.html#samplefilter

The SampleFilter sub-component is used for post-processing samples emitted by the decimator. Two implementations for the sample filter sub-component are provided by this library.

The NopSampleFilter class can be used to effectively disable per-sample filtering on the output of the decimator. It does nothing to the samples presented to it, and so calls to it can be optimized out during compilation.

The DcoeSampleFilter class is used for applying the DC offset elimination filter to the decimator’s output. The DC offset elimination filter is meant to ensure the sample mean for each channel tends toward zero.

For more details, see Sample Filters.

lib_mic_array: PDM microphone array library$$$Software Structure$$$Sub-Components$$$OutputHandler£££doc/rst/src/software_structure.html#outputhandler

The OutputHandler sub-component is responsible for transferring processed sample data to subsequent processing stages.

There are two main considerations for output handlers. The first is whether audio data should be transferred sample-by-sample or as frames containing many samples. The second is the method of actually transferring the audio data.

The class ChannelSampleTransmitter sends samples one at a time to subsequent processing stages using an xCore channel.

The FrameOutputHandler class collects samples into frames, and uses a frame transmitter to send the frames once they’re ready.

lib_mic_array: PDM microphone array library$$$Software Structure$$$Sub-Components$$$Prefabs£££doc/rst/src/software_structure.html#prefabs

One of the drawbacks to broad use of class templates is that concrete class names can unfortunately become excessively verbose and confusing. For example, the following is the fully qualified name of a (particular) concrete MicArray implementation:

mic_array::MicArray<2,
    mic_array::TwoStageDecimator<2,6,65>,
    mic_array::StandardPdmRxService<2,2,6>,
    mic_array::DcoeSampleFilter<2>,
    mic_array::FrameOutputHandler<2,256,
        mic_array::ChannelFrameTransmitter>>

This library also provides a C++ namespace mic_array::prefab which is intended to simplify construction of MicArray objects where common configurations are needed.

The BasicMicArray class template uses the most typical component implementations, where PDM rx can be run as an interrupt or as a stand-alone thread, and where audio frames are transmitted to subsequent processing stages using a channel.

To demonstrate how BasicMicArray simplifies this process, observe that the following MicArray type is behaviorally identical to the above:

mic_array::prefab::BasicMicArray<2,256,true>

lib_mic_array: PDM microphone array library$$$Mic Array Resource Usage$$$Discrete Resources$$$Ports£££doc/rst/src/resource_usage.html#ports

In all configurations, the mic array unit requires 3 of the xcore.ai device’s hardware ports. Two of these ports (for the master audio clock and PDM clock) must be 1-bit ports. The third (PDM capture port) can be 1-, 4- or 8-bit, depending on the microphone count and SDR/DDR configuration.

lib_mic_array: PDM microphone array library$$$Mic Array Resource Usage$$$Discrete Resources$$$Clock Blocks£££doc/rst/src/resource_usage.html#clock-blocks

In applications which use an SDR microphone configuration, the mic array unit requires 1 of the xcore.ai device’s 5 clock blocks. This clock block is used both to generate the PDM clock from the master audio clock and as the PDM capture clock.

In applications which use a DDR microphone configuration, the mic array unit requires 2 of the xcore.ai device’s 5 clock blocks. One clock is used to generate the PDM clock from the master audio clock, and the other is used as the PDM capture clock (which must operate at different rates in a DDR configuration).

lib_mic_array: PDM microphone array library$$$Mic Array Resource Usage$$$Discrete Resources$$$Chanends£££doc/rst/src/resource_usage.html#chanends

Chanends are a hardware resource which allow threads (possibly running on different tiles) to communicate over channels. The mic array unit requires 4 chanends. Two are used for communication between the PDM rx service and the decimation thread. Two more are needed for transfering completed frames from the mic array unit to other application components.

lib_mic_array: PDM microphone array library$$$Mic Array Resource Usage$$$Discrete Resources$$$Threads£££doc/rst/src/resource_usage.html#threads

The prefab API can run the PDM rx service either as a stand-alone thread or as an interrupt in another thread. The Vanilla API only supports running it as an interrupt. The Vanilla API requires only on hardware thread. The prefab API requires 1 thread if PDM rx is used in interrupt mode, and 2 if PDM rx is a stand-alone thread..

Running PDM rx as a stand-alone thread modestly reduces the mic array unit’s MIPS consumption by eliminating the context switch overhead of an interrupt. The cost of that is one hardware thread.

lib_mic_array: PDM microphone array library$$$Vanilla API$$$Configuration$$$mic_array_vanilla_add()£££doc/rst/src/vanilla_api.html#mic-array-vanilla-add

mic_array_vanilla_add() is the CMake macro used to add the Vanilla API to an application.

macro( mic_array_vanilla_add
          TARGET_NAME
          MCLK_FREQ
          PDM_FREQ
          MIC_COUNT
          SAMPLES_PER_FRAME )

TARGET_NAME

The name of the application’s CMake target. It is the target the Vanilla API is added to.

MCLK_FREQ

The known frequency, in Hz, of the application’s master audio clock. A typical frequency is 24576000 Hz. Note that this parameter is not configuring the master audio clock. (Equivalent compile definition: MIC_ARRAY_CONFIG_MCLK_FREQ)

PDM_FREQ

The desired frequency, in Hz, of the PDM clock. This should be an integer factor of MCLK_FREQ between 1 and 510. (Equivalent compile definition: MIC_ARRAY_CONFIG_PDM_FREQ)

MIC_COUNT

The number of PDM microphone channels to be captured. This API supports values of 1 (SDR), 2 (DDR), 4 (SDR) and 8 (SDR/DDR). This value must match the configuration (SDR/DDR) and port width of the PDM capture port. That is, in an SDR port configuration, MIC_COUNT must equal the capture port width, and in DDR port configuration, MIC_COUNT must be twice the port width. (Equivalent compile definition: MIC_ARRAY_CONFIG_MIC_COUNT)

SAMPLES_PER_FRAME is the number of samples (for each microphone channel) that will be delivered in each (non-overlapping) frame retrieved by ma_frame_rx(). A minimum value of 1 is supported, to deliver samples one at a time. The larger this value, the looser the real-time constraint on the thread receiving the mic array unit’s output (while also increasing the amount of audio data to be processed).

lib_mic_array: PDM microphone array library$$$Vanilla API$$$Configuration$$$Optional Configuration£££doc/rst/src/vanilla_api.html#optional-configuration

These are configuration parameters that receive default values but can be optionally overridden by an application. These can be defined in your application’s CMakeLists.txt using CMake’s built-in target_compile_definitions() command.

MIC_ARRAY_CONFIG_USE_DDR

Indicates whether the microphones are arranged in an SDR (0) or DDR (1) configuration. An SDR configuration is one in which each port pin is connected to a single PDM microphone. A DDR configuration is one which each port pin is connected to two PDM microphones. Defaults to 0 (SDR), unless MIC_ARRAY_CONFIG_MIC_COUNT is 2 in which case it defaults to 1 (DDR).

MIC_ARRAY_CONFIG_USE_DC_ELIMINATION

Indicates whether the DC offset elimination filter should be applied to the output of the decimator. Set to 0 to disable or 1 to enable. Defaults to 1 (filter on).

The next three parameters are the identifiers for hardware port resources used by the mic array unit. They can be specified as either the identifier listed in your device’s datasheet (e.g. XS1_PORT_1D) or as an alias for the port listed in your application’s XN file (e.g. PORT_MCLK_IN_OUT). For example:

...
<Tile Number="0" Reference="tile[0]">
...
  <Port Location="XS1_PORT_1D"  Name="PORT_MCLK_IN_OUT"/>
...
</Tile>
...

MIC_ARRAY_CONFIG_PORT_MCLK

Identifier of the 1-bit port on which the device is receiving the master audio clock. Defaults to PORT_MLCK_IN_OUT.

MIC_ARRAY_CONFIG_PORT_PDM_CLK

Identifier of the 1-bit port on which the device will signal the PDM clock to the microphones. Defaults to PORT_PDM_CLK.

MIC_ARRAY_CONFIG_PORT_PDM_DATA

Identifier of the port on which the device will capture PDM sample data. The port width of this port must match the MIC_COUNT parameter given to mic_array_vanilla_add() and the value of MIC_ARRAY_CONFIG_USE_DDR. Defaults to PORT_PDM_DATA.

The final two parameters indicate which clock block resource(s) should be used to generate the PDM clock and the capture clock. An xcore.ai device provides 5 hardware clock blocks for application use, identified as XS1_CLKBLK_1 through XS1_CLKBLK_5. The device’s clock blocks are interchangeable, but if another component of your application uses one of these defaults, you may need to change these parameters.

MIC_ARRAY_CONFIG_CLOCK_BLOCK_A

Clock block used as ‘clock A’ (see Getting Started). This clock block is used in both SDR and DDR configurations.

MIC_ARRAY_CONFIG_CLOCK_BLOCK_B

Clock block used as ‘clock B’ (see Getting Started). This clock block is only needed in DDR configurations and is ignored (not configured) in SDR configurations.

lib_mic_array: PDM microphone array library$$$Vanilla API$$$Configuration$$$Vanilla API with other Build Systems£££doc/rst/src/vanilla_api.html#vanilla-api-with-other-build-systems

Using the Vanilla API with build systems other than CMake is simple.

• Add the file etc/vanilla/mic_array_vanilla.cpp to the application’s source files.

• Add etc/vanilla/ (relative to repository root) to the application include paths.

• Add the compile definitions for the parameters listed in the previous sections (each parameter beginning with MIC_ARRAY_CONFIG_) to the compile options for mic_array_vanilla.cpp.