Hide navigation sidebar
Hide table of contents sidebar
Back to top
Toggle table of contents sidebar

# Modifying the Software

## Implementing the ASR API

Begin your ASR port by creating a new folder under `modules/asr/`. The `asr.h` and `device_memory.h` files include comments detailing the public API methods and parameters. ASR ports that implement the public API defined can easily be added to current and future XCORE-VOICE example designs that support speech recognition.

Pay close attention to the functions: - `asr_printf` - `devmem_malloc` - `devmem_free` - `devmem_read_ext` - `devmem_read_ext_async` - `devmem_read_ext_wait`

ASR libraries should call `asr_printf` instead of `printf` or xcore’s `debug_printf`.

ASR libraries must not call `malloc` directly to allocate dynamic memory. Instead call the `devmem_malloc` and `devmem_free` functions. This allows the application to provide alternative implementations of these functions - like `pvPortMalloc` and `vPortFree` in a FreeRTOS application.

The `devmem_read_ext` function is provided to load data directly from external memory (QSPI flash or LPDDR) into SRAM. This is the recommended way to load coefficients or blocks of data from a model. It is far more efficient to load the data into SRAM and perform any math on the data while it is in SRAM. The `devmem_read_ext` function a signature similar to `memcpy`. The caller is responsible for allocating the destination buffer.

Like `devmem_read_ext`, the `devmem_read_ext_async` function is provided to load data directly from external memory (QSPI flash or LPDDR) into SRAM. `devmem_read_ext_async` differs in that it does not block the caller’s thread. Instead it loads the data in another thread. One must have a free core when calling `devmem_read_ext_async` or an exception will be raised. `devmem_read_ext_async` returns a handle that can later be used to wait for the load to complete. Call `devmem_read_ext_wait` to block the callers thread until the load is complete. Currently, each call to `devmem_read_ext_async` must be followed by a call to `devmem_read_ext_wait`. You can not have more than one read in flight at a time.

Note

XMOS provides an arithmetic and DSP library which leverages the XS3 Vector Processing Unit (VPU) to accelerate costly operations on vectors of 16- or 32-bit data. Included are functions for block floating-point arithmetic, fast Fourier transforms, discrete cosine transforms, linear filtering and more. See the XMath Programming Guide for more information.

Note

To minimize SRAM scratch space usage, some ASR ports load coefficients into SRAM in chunks. This is useful when performing a routine such as a vector matrix multiply as this operation can be performed on a portion of the matrix at a time.

When the port of the new ASR is complete, you can use the example in `examples/speech_recognition` to test it.

Note

You may also need to modify `BRICK_SIZE_SAMPLES` in `app_conf.h` to match the number of audio samples expected per process for your ASR port. In other example designs, this is defined by `appconfINTENT_SAMPLE_BLOCK_LENGTH`. This is set to 240 in the existing example designs.

In the current source code, the model data (and optional grammar data) are set in `examples/speech_recognition/src/process_file.c`. Modify these variables to reflect your data. The remainder of the API should be familiar to ASR developers. The API can be extended if necessary.

## Flashing Models

To flash your model, modify the `--data` argument passed to `xflash` command in the Flashing the Model section.

See `examples/speech_recognition/asr_example/asr_example_model.h` to see how the model’s flash address is defined.

## Placing Models in SRAM

Small models (near or under 100kB in size) may be placed in SRAM. See `examples/speech_recognition/asr_example/asr_example_model.c` for more information on placing your model in SRAM.

# ASR API

enum asr_error_enum

  Enumerator type representing error return values.

  Values:

  enumerator ASR_OK

    Ok.

  enumerator ASR_ERROR

    General error.

  enumerator ASR_INSUFFICIENT_MEMORY

    Insufficient memory for given model.

  enumerator ASR_NOT_SUPPORTED

    Function not supported for given model.

  enumerator ASR_INVALID_PARAMETER

    Invalid Parameter.

  enumerator ASR_MODEL_INCOMPATIBLE

    Model type or version is not compatible with the ASR library.

  enumerator ASR_MODEL_CORRUPT

    Model malformed.

  enumerator ASR_NOT_INITIALIZED

    Not Initialized.

  enumerator ASR_EVALUATION_EXPIRED

    Evaluation period has expired.

typedef void *asr_port_t

  Typedef to the ASR port context struct.

  An ASR port can store any data needed in the context. The context pointer is passed to all API methods and can be cast to any struct defined by the ASR port.

typedef int16_t asr_sample_t

  Typedef representing the base type of an audio sample.

typedef struct asr_attributes_struct asr_attributes_t

  Typedef to the ASR port and model attributes

typedef struct asr_result_struct asr_result_t

  Typedef to the ASR result

typedef enum asr_error_enum asr_error_t

  Enumerator type representing error return values.

void asr_printf(const char *format, ...)

  String output function that allows the application to provide an alternative implementation.

  ASR ports should call asr_printf instead of printf

asr_port_t asr_init(int32_t *model, int32_t *grammar, devmem_manager_t *devmem_ctx)

  Initialize an ASR port.

  Parameters:

    - model – A pointer to the model data.

    - grammar – A pointer to the grammar data (Optional).

    - devmem_ctx – A pointer to the device manager (Optional). Save this pointer if calling any device manager API functions.

  Returns:

    the ASR port context.

asr_error_t asr_get_attributes(asr_port_t *ctx, asr_attributes_t *attributes)

  Get engine and model attributes.

  Parameters:

    - ctx – A pointer to the ASR port context.

    - attributes – The attributes result.

  Returns:

    Success or error code.

asr_error_t asr_process(asr_port_t *ctx, int16_t *audio_buf, size_t buf_len)

  Process an audio buffer.

  Parameters:

    - ctx – A pointer to the ASR port context.

    - audio_buf – A pointer to the 16-bit PCM samples.

    - buf_len – The number of PCM samples.

  Returns:

    Success or error code.

asr_error_t asr_get_result(asr_port_t *ctx, asr_result_t *result)

  Get the most recent results.

  Parameters:

    - ctx – A pointer to the ASR port context.

    - result – The processed result.

  Returns:

    Success or error code.

asr_error_t asr_reset(asr_port_t *ctx)

  Reset ASR port (if necessary).

  Called before the next call to asr_process.

  Parameters:

    - ctx – A pointer to the ASR port context.

  Returns:

    Success or error code.

asr_error_t asr_release(asr_port_t *ctx)

  Release ASR port (if necessary).

  The ASR port must deallocate any memory.

  Parameters:

    - ctx – A pointer to the ASR port context.

  Returns:

    Success or error code.

struct asr_attributes_struct
  #include <asr.h>

  Typedef to the ASR port and model attributes

struct asr_result_struct
  #include <asr.h>

  Typedef to the ASR result

# Device Memory API

void *devmem_malloc(devmem_manager_t *ctx, size_t size)

  Memory allocation function that allows the application to provide an alternative implementation.

  Call devmem_malloc instead of malloc

  Parameters:

    - ctx – A pointer to the device memory context.

    - size – Number of bytes to allocate.

  Returns:

    A pointer to the beginning of newly allocated memory, or NULL on failure.

void devmem_free(devmem_manager_t *ctx, void *ptr)

  Memory deallocation function that allows the application to provide an alternative implementation.

  Call devmem_free instead of free

  Parameters:

    - ctx – A pointer to the device memory context.

    - ptr – A pointer to the memory to deallocate.

void devmem_read_ext(devmem_manager_t *ctx, void *dest, const void *src, size_t n)

  Synchronous extended memory read function that allows the application

  to provide an alternative implementation. Blocks the callers thread until the read is completed.

  Call devmem_read_ext instead of any other functions to read memory from flash, LPDDR or SDRAM. Modules are free to use memcpy if the dest and src are both SRAM addresses.

  Parameters:

    - ctx – A pointer to the device memory context.

    - dest – A pointer to the destination array where the content is to be read.

    - src – A pointer to the word-aligned address of data to be read.

    - n – Number of bytes to read.

int devmem_read_ext_async(devmem_manager_t *ctx, void *dest, const void *src, size_t n)

  Asynchronous extended memory read function that allows the application

  to provide an alternative implementation.

  Call asr_read_ext_async instead of any other functions to read memory from flash, LPDDR or SDRAM.

  Parameters:

    - ctx – A pointer to the device memory context.

    - dest – A pointer to the destination array where the content is to be read.

    - src – A pointer to the word-aligned address of data to be read.

    - n – Number of bytes to read.

  Returns:

    A handle that can be used in a call to devmem_read_ext_wait.

void devmem_read_ext_wait(devmem_manager_t *ctx, int handle)

  Wait in the caller’s thread for an asynchronous extended memory read to finish.

  Parameters:

    - ctx – A pointer to the device memory context.

    - handle – The devmem_read_ext_asyc handle to wait on.

IS_SRAM(a)

IS_SWMEM(a)

IS_FLASH(a)