On i.MX8 k_cycle_get_64 always returns 0 so use sof_cycle_get_64
instead.
There is no better solution to support 64bit cycle counting for i.MX
and will have to stick with this implementation.
Fixes: c3c94fc515 ("header: rtos: use rtos specific version of wait.h")
Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>
Define a dedicated .heap section in case of zephyr build.
Such explicit section is required to allow for unmapping
of unused physical pages which can be later used for
loadable library/modules support.
Signed-off-by: Rafal Redzimski <rafal.f.redzimski@intel.com>
Continue changes in commit c194125b83 ("zephyr: add
notifier_register(ipc_send_queued_msg) in task_main_start()") and
implement the delayed IPC msg sending in Zephyr work queue.
The old implementation did not cover cases where IPC needed to be resent
when no active LL scheduler tasks were active.
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Code can now include <rtos/alloc.h> and uses thinly wrapped Zephyr
native for most uses. Wrapping can be removed over time.
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Code can now include <rtos/interrupt.h> and uses native Zephyr IRQ API
for most uses. Some wrapping still exists which can be removed over
time. IMX is missing IRQ driver in Zephyr so falls back to xtos IRQ
driver.
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add IPC4 support for extraction & injection probes
IPC4 header changes, checksum calculation
moved checksum from header to the end of data packet
Signed-off-by: Adrian Bonislawski <adrian.bonislawski@intel.com>
Code can now include <rtos/wait.h> and uses native Zephyr 64
cycle API instead of SOF version.
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
With Zephyr, SOF uses k_cycle_get_64() API in order to
count clock cycles.
Not all platforms have a 64bit counter (e.g i.MX dsp integration
only has xtensa timer which is limited to 32 bits).
So, instead of using k_cycle_get_64 introduce new API sof_cycle_get_64
which keeps the existing behavior for platforms which define
CONFIG_TIMER_HAS_64BIT_CYCLE_COUNTER, otherwise use k_uptime_ticks().
See comments from @andyross at
https://github.com/zephyrproject-rtos/zephyr/pull/48318
Suggested-by: Andy Ross <andrew.j.ross@intel.com>
Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>
Use the module interface instead of the comp drv interface for the
EQ IIR component.
Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
Add multiple third-party and optional audio processing components to
the Zephyr cmake and initialisation lists.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Move the volume component inside the module_adapter folder and use the
module interface API instead of the component driver interface. Also fix
the volume cmocka tests to use the modified scale_vol function.
Also, add kconfig option for keeping the legacy comp_drv interface for platforms
that are limited in text region memory size, such as
BYT/CHT/APL/JSL/ICL.
Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
heap_alloc_aligned_cached() is a helper function used to allocate
memory with a cached alias. Therefore its returned pointer should
use the __sparse_cache annotation.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
to DECLARE_MODULE_ADAPTER and comp_driver to comp_module_adapter.
Also rename SOF_COMP_MODULE_ADAPTER to SOF_COM_MODULE_ADAPTER.
No functional changes.
Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com>
The log context in zephyr is per file or module.
To use zephyr logging api, LOG_MODULE_REGISTER is used
to register a log context, LOG_MODULE_DECLARE is used
to refer to the registered context.
For function in header file, LOG_MODULE_DECLARE should
be used within that function to avoid context collapse,
a condition one source file have multiple context
registered or declared.
Signed-off-by: Chao Song <chao.song@linux.intel.com>
Add a static variable w_core_enable_mask in Zephyr wrapper.c to
track requests to enable and disable cores, and use this to
implement arch_cpu_enabled_cores().
We don't yet have a implementation to power off secondary cores.
This creates issues to SOF application code that may rely on
disabling of cores to be reflected in cpu_enabled_cores(). Instead
of adding ifdefs in application code, manage this within wrapper.c.
Once the secondary core power-off is implemented, this static
variable can be removed.
BugLink: https://github.com/thesofproject/sof/issues/5701
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The function rballoc_align did not take into account the passed flags, so
it always incorrectly allocated a cached memory. Added support for the
SOF_MEM_FLAG_COHERENT flag.
Signed-off-by: Adrian Warecki <adrianx.warecki@intel.com>
`_end` and `_heap_sentry` linker symbols are used to infer heap size. In
Zephyr, `_end` is referenced as `char []` while in SOF as `uint8_t`.
This may "conflicting types", should those declarations be built
together.
This patch changes the Zephyr wrapper to also use `char []`, thus
avoiding this issue.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Multi-core configurations already use uncached memory for LLI. This
also makes sense to avoid having to manually synchronise cache. Force
LLI objects uncached.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Replaced sof timer related functions with Zephyr alternative.
Signed-off-by: Adrian Warecki <adrianx.warecki@intel.com>
Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>
Fix issue of ipc4 pipeline initialization with zephyr enabled.
Need to initialize these two components. Gain module is another
type of volume without peak volume support and mixin has subset
function of mixer.
Signed-off-by: Rander Wang <rander.wang@intel.com>
Use the new interface to initialize non-primary cores before
enabling them with arch_start_cpu(). This interface is not
available in Zephyr 3.0.0, but is mandatory in later release.
Depends on Zephyr commit 2b210cb3db05 ("kernel: Refactor SMP cpu
initialization a bit").
BugLink: https://github.com/thesofproject/sof/issues/5456
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
z_wrapper_cpu_enable_secondary_core() may be called
repeatedly on a core that is already running. Add a check
to ensure we do not try to restart a core that is already
running.
Also add a check for ready_flag as done in upstream Zephyr
kernel/smp.c to ensure z_wrapper_cpu_enable_secondary_core()
does not return until the target core has finished its
startup and arch_cpu_active() returns true.
BugLink: https://github.com/thesofproject/sof/issues/5446
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
This enables codec_adapter and cadence/passhtrough codecs
with SOF on Zephyr.
passthrough codec can be used as it is, but cadence codec
will also need codec libraries from cadence.
Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>
The two components copier and base_fw are
essential to IPC4 support on SOF on Zephyr.
This patch integrates the build of the two
components to SOF build with zephyr RTOS.
Signed-off-by: Chao Song <chao.song@linux.intel.com>
To align with Zephyr smp.c:smp_init_top(), add the missing call to
smp_timer_init() in secondary_init() of the SOF wrapper. Also
add inline documentation why we are open-coding the function on
SOF side.
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
This reverts commit 414089aa44.
Failures hit with 4core cAVS2.5 configuraions, so this has
to be reverted. We need an implementation that keeps both
Zephyr (and SOF runtime-DSP if still used) in the loop.
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Currently there is code to power up secondary cores both
in Zephyr architecture hooks in wrapper.c, as well as in
SOF runtime-PM driver.
Some of this is necessary overlap until full replacement of
SOF runtime-PM is available. But for secondary core power up,
we can already clean up the flow and hook the Zephyr core
enable code to runtime-PM driver directly. This fixes startup
issues seen on Intel cAvS2.5 platforms.
Core power-down is left intentionally for runtime-PM driver
to handle as the related interface in Zephyr is not ready
for use yet.
BugLink: https://github.com/thesofproject/sof/issues/5018
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
This reverts commit 079ce95b98.
As explained in the comment that it modified, predicted in the #5159
discussion and reproduced in test #5290 /
https://github.com/thesofproject/sof/runs/4988844402, this commit broke
'CONFIG_TRACE=n'. Simple reproduction steps:
$ rm -rf zephyrproject/build-apl
$ echo 'CONFIG_TRACE=n' >> \
boards/xtensa/intel_adsp_cavs15/intel_adsp_cavs15_defconfig
$ sof/scripts/xtensa-build-zephyr.sh [ -p /path/to/zephyrproject ] apl
fw abi main version: 3.20.1
fw abi dbg version: 5.3.0
warning: can't find section .static_log_entries in zephyr/zephyr.elf
error: section .static_log_entries can't be found
error: unable to write dictionaries, -22
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
_smex_placeholder dowsn't have to be global, make it static.
Also no need to explicitly specify smex_placeholder_f() as inline,
the compiler will decide by itself.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
On generic Zephyr platform case, use an OS timer call to get a time
stamp.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
When built with Zephyr, SOF ideally should be a fully platform-
agnostic application. All platform-specific code should reside in
Zephyr itself, including core architecture and platform
infrastructures and drivers. To prepare for this transition we add a
generic SOF-Zephyr platform. It is based on the current Tiger Lake
architecture and should be usable for cAVS 2.5 and newer systems.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Primary core during going to D0i3 state disables all another cores
in platform_pg_int_handler() function. Before it secondary cores
should be prepared (disable interrupts, perform writeback) in order
to make proper restore flow after D0i3->D0 transition.
This commit adds cpu_secondary_cores_prepare_d0ix() function in
ipc_pm_gate() handler, which sends idc to secondary cores
with information that they should perform preparation for power down
in the next platform_wait_for_interrupt() invocation. In
platform_wait_for_interrupt() there is cpu_power_down_core()
invocation with CPU_POWER_DOWN_MEMORY_ON (performs writeback/invalidate,
disables interrupts), when proper flag is set.
Signed-off-by: Bartosz Kokoszko <bartoszx.kokoszko@linux.intel.com>
This will add missing secondary cores restore on wake up,
simple pm_runtime_get(PM_RUNTIME_DSP, PLATFORM_PRIMARY_CORE_ID)
in lps is not enough in multicore scenario
Signed-off-by: Adrian Bonislawski <adrian.bonislawski@intel.com>
Instead of hard-coding a guessed heap size, obtain remaining free
SRAM from the linker and use it completely for the heap.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Recent changes increased the size of various sections, breaking
linkage again. We need to make the linker calculate the heap size
automatically but until that is ready, reduce the heap size further
to fix the current breakage.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
As per documentation in include/sof/lib/alloc.h, rmalloc from
MEM_ZONE_SYS will always succeed. In the unlikely case the alloc
fails, trigger a system panic in rmalloc().
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Add caching to more heap zones. With addition of SYS, SYS_RUNTIME
and RUNTIME the mapping is now aligned with XTOS allocator.
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The XTOS allocator aligns non-cachec allocations to the platform
cacheline size. Test results indicate some SOF application code
relies on this behaviour, so align Zephyr's rmalloc implementation
to match the behaviour.
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Implement the cached SOF heap zones using a single sys_heap
object.
Compared to old implementation which used two separate sys_heaps,
this implementation enables dynamic cached/uncached partitioning
of the heap.
To ensure coherency of the Zephyr heap operation, if cached heap
is enabled, all allocations must be aligned to dcache linesize.
Add a Kconfig option CONFIG_SOF_ZEPHYR_HEAP_CACHED to turn on/off
the cached heap logic, with default set to no for all platforms.
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The heap code was invalidating blocks on allocation, but that's the
wrong side of the pipe. By definition, new heap memory
will/should/must be written before being used (because the memory is
undefined), so any cached contents are irrelevant as they'll be
overwritten.
But when the user is finished with the block and frees it, there may
still be live dirty cache lines in the region on the current CPU.
Those must be invalidated, otherwise they will be evicted from the
cache at some point in the future, on top of the memory region now
being used for different purposes on another CPU.
Remove the invalidate on allocation. Add it back in free. Leverage a
new Zephyr sys_heap_usable_size() routine to get the size so we don't
have to store it in an extra header.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Building SOF with Zephyr for Apollolake with XCC fails because of
insufficient RAM. Reduce the heap size to fix the problem.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
We have change the size of runtime_shared zone to be decided at link
stage, here hardcode HEAP_RUNTIME_SHARED_SIZE to unblock the Zephyr
building.
Signed-off-by: Keyon Jie <yang.jie@linux.intel.com>
sof_heap variable is used by all platform, with or
without ENABLE_CACHED_HEAP.
Without this fix, on i.MX with Zephyr we get:
error: 'sof_heap' undeclared (first use in this function);
Fixes: fed69b6a34 ("zephyr: wrapper: disable cached heap on Intel platforms")
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
Tests with dynamic-pipeline topologies increase the complexity
of alloc/free patterns for heap and these tests have uncovered
new issues with heap corruption. Tests are reporting failed
allocs although heap still has free space. Running same test
with cached heap disabled shows no failures.
The cached heap is already disabled for IMX targets, so build on
this code and add a local ENABLE_CACHED_HEAP define to toggle
the cached heap.
BugLink: https://github.com/thesofproject/sof/issues/4494
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Clean up the ifdefs for ipc_pm_context_save(). By implementing
platform_timer_stop() in the Zephyr wrapper, the original context
save sequence can be used also for Zephyr builds.
By doing this, we can fix issues with IPC timeout on CTX_SAVE.
Disabling all interrupts before the IPC reply is needed also on
Zephyr.
BugLink: https://github.com/thesofproject/sof/issues/4507
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
In XTOS SOF, ipc_send_queued_msg() is run by task_main_primary_core().
In Zephyr we need to schedule ipc_send_queued_msg() using a notifier
triggered by the periodic ll_scheduler.
This is similar to commit c194125b83 ("zephyr:
add notifier_register(ipc_send_queued_msg) in task_main_start()")
For i.MX we need to use this temporary fix for SOF_SCHEDULE_LL_DMA, also.
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
On i.MX the DMA interrupts are routed via IRQ_STEER.
In order for this to work we need to:
- make any second level interrupts handling go
through interrupt-irqsteer.c;
- use first level interrupt handling from
wrapper.c.
TODO: Implement a driver for the IRQ_STEER in Zephyr,
to replace the legacy code (interrupt-irqsteer.c).
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
Commit 34bb9b7282 ("zephyr: wrapper: build SOF with Zephyr for imx")
broke coherent memory allocations on Intel DSP platforms. Restore the
original code.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
When building SOF with Zephyr, for i.MX, we get the following
error: undefined reference to `z_xtensa_cache_inv'.
i.MX has only one DSP core, so CONFIG_KERNEL_COHERENCE is not set.
Therefore include arch/xtensa/cache.h, where z_xtensa_cache_inv
is defined and implemented, in cases when the kernel is not built in
a mode where all shared data is placed in uncached memory.
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
When building SOF with Zephyr for i.MX the following
additional adjustments, to the wrapper, are required:
1. include heapmem variable in .heap_mem section,
otherwise the HEAP_SIZE is duplicated in two sections and
the sdram0 region overflows;
2. no need to split heap into shared and unshared since
we only have 1 DSP core; In this case, the kernel will
never be built in a mode where all shared data is placed
in multiprocessor-coherent (generally "uncached") memory.
3. use simple interrupt_get_irq() to get the Linux interrupt
and later pass it to irq_steer;
4. use Xtensa timer, as we do now with SOF and XTOS.
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
Fixes#4356
In XTOS SOF, ipc_send_queued_msg() is run by task_main_primary_core(),
the initialization of which has a hard and circular dependency with the
initialization of the EDF scheduler. This EDF scheduler is not part of
Zephyr. Because of the circular dependency it does not seem
possible (nor desirable?) to refactor this code and make it compatible
with Zephyr. So schedule ipc_send_queued_msg() differently in Zephyr:
using a notifier triggered by the periodic Low Latency scheduler.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
To increase performance of audio application code, the SOF heap should
be mapped to cached address region on platforms with aliased
cached/uncached memory layout.
This splits the heap into two and maps the SOF_MEM_ZONE_BUFFER to
the cached heap and all rest to uncached heap.
The raw heap memory area is still mapped to uncached memory, even for
the cached heap. The allocated blocks are converted to cached addresses
and passed to SOF users of rmalloc() and rballoc_align(). To ensure
application buffers do not share cachelines with heap metadata chunks,
allocation sizes are rounded up to integer multiples of platform
cacheline size.
BugLink: https://github.com/thesofproject/sof/issues/4306
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The key-phrase-buffer (kpb) component can be included in
Zephyr-based builds, but the component is not initialized
properly in the builds.
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Notifier data has to be per-core and its allocation is
performed in init_system_notify(), no need to also do it in
architecture code.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
sof_main() doesn't run on secondary cores. Instead we provide
a new function, that initialises a dummy thread, runs SOF
initialisation by calling secondary_core_init() and switches
over to the dummy thread.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Move logging code and static data to platform/library and
declare a LOG module properly.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
The use of Zephyr LOG_*() API in SOF is breaking compilation.
Replace it with with the native trace logging until a proper
fix is available.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
SOF registers IRQ handlers dynamically, therefore disabling
support for dynamic interrupts for SOF builds doesn't make
much sense. However, this can be used to at least increase
compile-time testing coverage and potentially enable
applications, not using dynamic interrupt registration.
BugLink: https://github.com/thesofproject/sof/issues/4044
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
The Zephyr timer implementation reads high and low 32 bits
in a non-atomic way. Add a loop to make sure the read is
consistent.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Recently added platform_timer_get_atomic() has to be
implemented for Zephyr too.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Add some files from src/platform/library/ to generic arch
builds with Zephyr.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
When building SOF with Zephyr for unsupported architectures
the following additional adjustments to the wrapper are required:
1. define HEAP_RUNTIME_SIZE
2. implement stubs for interrupt handlich code
3. remove arch_timer_get_system() as it's provided by a header
4. hardcode "return 0;" for platform_timer_get()
5. disable sys_module_init() which anyway isn't yet functional
under Zephyr
6. add dummy platform_init() and platform_boot_complete()
7. add log_const_sof
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Broadwell support with Zephyr had only been dry-coded, it has never
been tested, and it contains an infinite recursion bug: functions
platform_timer_get() and arch_timer_get_system() call each other
infinitely. Break the recursion formally, until a proper fix is
available.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
This kconfig variable no longer exists in upstream Zephyr (the feature
it used to control is now zero-overhead and always enabled).
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Audio component samples have been moved to a new location, fix Zephyr
builds to account for it.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
This patch finalises the wrapper to allow LL scheduling of
timer domain audio, EDF sccheduling of IPC, trace re-direction
and kconfig build support.
There are still many items that are WIP and have been marked with
TODO: comments.
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add a wrapper to convert SOF API calls to Zephyr equivalents. This allows
development to continue in parallel with xtos/HAL and Zephyr RTOSes until
parity is reached.
The initlial Zephyr APIs to be used are HAL, boot, IRQs, scheduling and
memory. The wrapper reflects this and will probably grow until feature
parity is reached when it can be deleted (as Zephyr APIs can be used
directly).
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Daniel Baluta