Examples

This section goes through the various example applications that are provided alongside the lib_voice. The examples are meant to demonstrate the basic usage of the APIs. All examples are located in the examples directory.

AEC example

The AEC example demonstrates the use of API to run a frame of data through the AEC. It also shows how to configure AEC to run on one or two threads. AEC_THREADS define toggles the AEC to use 1-threaded or 2-threaded scheduling structs:

    #if AEC_THREADS == 1
    aec_task_distribution_t tdist = aec_tdist_chans2_threads1;
    #elif AEC_THREADS == 2
    aec_task_distribution_t tdist = aec_tdist_chans2_threads2;
    #else
    #error "Unsupported number of AEC threads"
    #endif

The build system will automatically create both configs called app_aec_1th and app_aec_2th.

After the application has decided which thread distribution scheme to run, it allocates memory for the AEC and initialises it:

    // Allocate signal data
    int32_t DWORD_ALIGNED frame_y[AEC_MAX_Y_CHANNELS][AEC_FRAME_ADVANCE];
    int32_t DWORD_ALIGNED frame_x[AEC_MAX_X_CHANNELS][AEC_FRAME_ADVANCE];

    // Initialise AEC
    aec_state_t DWORD_ALIGNED aec_state;
    aec_init(&aec_state,
            AEC_MAX_Y_CHANNELS, AEC_MAX_X_CHANNELS,
            AEC_MAIN_FILTER_PHASES, AEC_SHADOW_FILTER_PHASES, &tdist);

After the AEC has been initialised, it is ready to process data. In this example, frame_y and frame_x memory is reused for AEC output:

        producer(frame_y, frame_x);
        // Reuse mic data memory for the main filter output
        // Reuse ref data memory for the shadow filter output
        aec_process_frame(&aec_state, frame_y, frame_x, frame_y, frame_x);
        consumer(frame_y);

Upon execution, the example will print “frame done” when the AEC has processed a frame.

VNR example

The VNR example demonstrates the use of API to run a frame of data through the VNR and get the estimation of how much voice is present in it.

To run the VNR, the application should first allocate memory for its data and initialise it:

  // Allocate input and output memory
  int32_t input[VNR_FRAME_ADVANCE] = {0};
  float_s32_t vnr_out;

  // Initialise VNR
  vnr_state_t vnr;
  vnr_state_init(&vnr);

After the VNR has been initialised, it is ready to process data. The VNR output is in float_s32_t format, so there’s an extra step if the user wants it in float.

    producer(input);
    vnr_process_frame(&vnr, &vnr_out, input);
    float res = float_s32_to_float(vnr_out);
    consumer(res);

Upon execution, the example will use the pseudo-random generator to get an input data. This data will be run through the VNR and the score will be printed in the terminal. It outputs a number between 0 and 1, 1 being the strongest voice with respect to noise and 0 being the lowest voice compared to noise ratio. The pseudo-random data is not representative of a real signal, so the VNR scores in this example tend to be zero.

Pipeline example

This example demonstrates how to put together a voice processing pipeline. The pipeline will have AEC, IC, NS and AGC stages. It also demonstrates the use of ADEC module to do a one time estimation and correction for possible reference and loudspeaker delay offsets at start up in order to maximise AEC performance. ADEC processing happens on the same thread as the AEC. The VNR is introduced to give the IC and the AGC information about the speech presence in a frame.

There are two pipelines supported in this example: Standard Architecture and Alternating Architecture. Building one or the other config is controlled via the ALT_ARCH_MODE define. The build system will automatically create both configs called app_pipeline_std_arch and app_pipeline_alt_arch.

To create the pipeline, the application must first initialise all the individual components:

    // Initialise AEC, DE, ADEC stages
    aec_conf_t aec_de_mode_conf, aec_non_de_mode_conf;
#if ALT_ARCH_MODE
    aec_non_de_mode_conf.num_y_channels = 1;
    aec_non_de_mode_conf.num_x_channels = AP_MAX_X_CHANNELS;
    aec_non_de_mode_conf.num_main_filt_phases = 15;
    aec_non_de_mode_conf.num_shadow_filt_phases = AEC_SHADOW_FILTER_PHASES;
#else
    aec_non_de_mode_conf.num_y_channels = AP_MAX_Y_CHANNELS;
    aec_non_de_mode_conf.num_x_channels = AP_MAX_X_CHANNELS;
    aec_non_de_mode_conf.num_main_filt_phases = AEC_MAIN_FILTER_PHASES;
    aec_non_de_mode_conf.num_shadow_filt_phases = AEC_SHADOW_FILTER_PHASES;
#endif
    aec_non_de_mode_conf.tdist = &aec_tdist_chans2_threads1;

    aec_de_mode_conf.num_y_channels = 1;
    aec_de_mode_conf.num_x_channels = 1;
    aec_de_mode_conf.num_main_filt_phases = 30;
    aec_de_mode_conf.num_shadow_filt_phases = 0;
    aec_de_mode_conf.tdist = &aec_tdist_chans2_threads1;

    // Disable ADEC's automatic mode. We only want to estimate and correct for the delay at startup
    adec_config_t adec_conf;
    adec_conf.bypass = 1; // Bypass automatic DE correction
#if DISABLE_INITIAL_DELAY_EST
    // Do not force a DE correction cycle on startup
    adec_conf.force_de_cycle_trigger = 0;
#else
    // Force a delay correction cycle, so that delay correction happens once after initialisation.
    // Make sure this is set back to 0 after adec has requested a transition into DE mode once,
    // to stop any further delay correction (automatic or forced) by ADEC
    adec_conf.force_de_cycle_trigger = 1;
#endif
    stage1_init(&state->stage_1_state, &aec_de_mode_conf, &aec_non_de_mode_conf, &adec_conf);

    // Initialise IC, VNR
    ic_init(&state->ic_state);

    // Initialise NS
    ns_init(&state->ns_state);

    // Initialise AGC
    agc_config_t agc_conf_asr = AGC_PROFILE_ASR;
    agc_init(&state->agc_state, &agc_conf_asr);

After the pipeline has been initialised, the data can be run through it. All the modules in the example can run on separate threads. To exchange information between the pipeline stages the metadata struct will need to be created to be populated and consumed by the different modules.

    /** Stage1 - AEC, DE, ADEC*/
    // stage1 will not process the frame in-place,
    // since mic input is needed to overwrite the output in certain cases
    int32_t stage_1_out[AEC_MAX_Y_CHANNELS][AP_FRAME_ADVANCE];

    stage1_process_frame(&state->stage_1_state, &stage_1_out[0], &md.max_ref_energy,
            &md.aec_corr_factor, &md.ref_active_flag, input_y_data, input_x_data);

    // Bypass IC if the reference is high in the alt arch mode
#if ALT_ARCH_MODE
    if(md.ref_active_flag) {
        state->ic_state.config_params.bypass = 1;
    }
    else {
        state->ic_state.config_params.bypass = 0;
    }
#endif
    /** IC and VNR*/
    int32_t ic_output[AP_FRAME_ADVANCE];
    float_s32_t input_vnr_pred;

    ic_process_frame(&state->ic_state, input_data[0], input_data[1], ic_output, &input_vnr_pred);
    md.vnr_pred_flag = input_vnr_pred;

    /** NS*/
    int32_t ns_output[AP_FRAME_ADVANCE];

    ns_process_frame(&state->ns_state, ns_output, ic_output);

    /** AGC*/
    agc_meta_data_t agc_md;
    agc_md.aec_ref_power = md.max_ref_energy;
    agc_md.vnr_flag = md.vnr_pred_flag;
    agc_md.ref_active_flag = md.ref_active_flag;
    agc_md.aec_corr_factor = md.aec_corr_factor;

    agc_process_frame(&state->agc_state, output_data, ns_output, &agc_md);

Upon execution, the example will print “frame done” when the AEC has processed a frame.

Building the example

This section assumes that the XMOS XTC Tools have been downloaded and installed. It also assumes that the Python is installed and available. The required versions are specified in the accompanying README.

Installation instructions can be found here.

Special attention should be paid to the section on Installation of Required Third-Party Tools.

The application is built using the xcommon-cmake build system, which is provided with the XTC tools and is based on CMake.

The lib_voice software ZIP package should be downloaded and extracted to a chosen working directory.

To configure the build, the following commands should be run from an XTC command prompt:

cd lib_voice
pip install -r requirements.txt
cd examples/app_aec
cmake -G "Unix Makefiles" -B build

Note

The pip install -r requirements.txt stage has to happen before the cmake configuration as it will fetch the xmos-ai-tools.

If any dependencies are missing they will be retrieved automatically during this step.

The application binaries should then be built using xmake:

xmake -j -C build

Binary artifacts (.xe files) will be generated under the appropriate subdirectories of the app_aec/bin directory — one for each supported build configuration.

For subsequent builds, the cmake step may be omitted. If CMakeLists.txt or other build files are modified, cmake will be re-run automatically by xmake as needed.

Running the example

From an XTC command prompt, the following command should be run from the examples/app_aec directory:

xrun --io ./bin/2th/app_aec_2th.xe