Open Profile for DICE is an open protocol for measured boot compatible
with the Trusted Computing Group's Device Identifier Composition
Engine (DICE) specification. The generated Compound Device Identifier
(CDI) certificates represent the hardware/software combination measured
by DICE, and can be used for remote attestation and sealing.
Add a driver that exposes reserved memory regions populated by firmware
with DICE CDIs and exposes them to userspace via a character device.
Userspace obtains the memory region's size from read() and calls mmap()
to create a mapping of the memory region in its address space. The
mapping is not allowed to be write+shared, giving userspace a guarantee
that the data were not overwritten by another process.
Userspace can also call write(), which triggers a wipe of the DICE data
by the driver. Because both the kernel and userspace mappings use
write-combine semantics, all clients observe the memory as zeroed after
the syscall has returned.
Acked-by: Rob Herring <robh@kernel.org>
Cc: Andrew Scull <ascull@google.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: David Brazdil <dbrazdil@google.com>
Link: https://lore.kernel.org/r/20220126231237.529308-3-dbrazdil@google.com
[dbrazdil@: Fixed context conflicts in reserved_mem_matches[] and Makefile]
Bug: 198197082
Change-Id: Iabd65f4d20036bb452e4103c7722f220c2273c81
(cherry picked from commit 8941930d125c97e7d22fb24eb10b431fad4436f4)
Add DeviceTree bindings for Open Profile for DICE, an open protocol for
measured boot. Firmware uses DICE to measure the hardware/software
combination and generates Compound Device Identifier (CDI) certificates.
These are stored in memory and the buffer is described in the DT as
a reserved memory region compatible with 'google,open-dice'.
'no-map' is required to ensure the memory region is never treated by
the kernel as system memory.
Bug: 198197082
Signed-off-by: David Brazdil <dbrazdil@google.com>
Link: https://lore.kernel.org/r/20220126231237.529308-2-dbrazdil@google.com
Change-Id: I6da35b90ca2f519408f96edf3999ffb257cf72e8
(cherry picked from commit 630c8e8d356745ebda5736da72c5734806de2c87)
This reverts commit d1109f05c3.
It will be replaced with the latest patch set version from upstream.
Bug: 198197082
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I661f1bc06d336e5eaab9e52affeb273a0ad6fc2f
(cherry picked from commit dbd93393826d11a10594182a33bd5c0b22d5378d)
This reverts commit 3d914125b2.
It will be replaced with the latest patch set version from upstream.
Bug: 198197082
Signed-off-by: David Brazdil <dbrazdil@google.com>
Change-Id: I8d9bc9dd059d316e5bc7ffbd0bb9a8a61fa770eb
(cherry picked from commit 011af7fd840c469c1b612763d43f6b3d78a1796d)
__migrate_task will not be present in non-smp builds, no need
to provide an inline function for that case.
Bug: 213581038
Fixes: 50f5345c87 ("ANDROID: __migrate_task header")
Change-Id: Ie8b8e07e4beaad7df169ac52169bd1799e610686
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
Correct a path to incremental-fs sysfs entry in incfs.rst
Bug: 211066171
Signed-off-by: Tadeusz Struk <tadeusz.struk@linaro.org>
Change-Id: Id3a94888edd9022c517939b4667d9792fc04146a
Syzbot recently found a number of issues related to incremental-fs
(see bug numbers below). All have to do with the fact that incr-fs
allows mounts of the same source and target multiple times.
The correct behavior for a file system is to allow only one such
mount, and then every subsequent attempt should fail with a -EBUSY
error code. In case of the issues listed below the common pattern
is that the reproducer calls:
mount("./file0", "./file0", "incremental-fs", 0, NULL)
many times and then invokes a file operation like chmod, setxattr,
or open on the ./file0. This causes a recursive call for all the
mounted instances, which eventually causes a stack overflow and
a kernel crash:
BUG: stack guard page was hit at ffffc90000c0fff8
kernel stack overflow (double-fault): 0000 [#1] PREEMPT SMP KASAN
The reason why many mounts with the same source and target are
possible is because the incfs_mount_fs() as it is allocates a new
super_block for every call, regardless of whether a given mount already
exists or not. This happens every time the sget() function is called
with a test param equal to NULL.
The correct behavior for an FS mount implementation is to call
appropriate mount vfs call for it's type, i.e. mount_bdev() for
a block device backed FS, mount_single() for a pseudo file system,
like sysfs that is mounted in a single, well know location, or
mount_nodev() for other special purpose FS like overlayfs.
In case of incremental-fs the open coded mount logic doesn't check
for abusive mount attempts such as overlays.
To fix this issue the logic needs to be changed to pass a proper
test function to sget() call, which then checks if a super_block
for a mount instance has already been allocated and also allows
the VFS to properly verify invalid mount attempts.
Bug: 211066171
Bug: 213140206
Bug: 213215835
Bug: 211914587
Bug: 211213635
Bug: 213137376
Bug: 211161296
Signed-off-by: Tadeusz Struk <tadeusz.struk@linaro.org>
Change-Id: I66cfc3f1b5aaffb32b0845b2dad3ff26fe952e27
It is possible that fget returns NULL. This needs to be handled
correctly in ioctl_permit_fill.
Bug: 212821226
Signed-off-by: Tadeusz Struk <tadeusz.struk@linaro.org>
Change-Id: Iec8be21982afeab6794b78ab1a542671c52acea2
In some cases we would like to bypass oom panic and give the system more
time to cleanup memory. Add vendor hook to allow skipping the oom panic.
Bug: 186875166
Change-Id: I64e74b2c013d6f18d74504777c6559d9ae07e008
Signed-off-by: Prakash Gupta <guptap@codeaurora.org>
Signed-off-by: Sudarshan Rajagopalan <quic_sudaraja@quicinc.com>
This reverts commit bf2290a48a (Revert "ANDROID: vendor_hooks: set
debugging data when rt_mutex is working")
The original patch has been reverted to resolve merge issues.
This patch adds again the vendor hooks for the original purpose.
Bug: 216016261
Signed-off-by: Sangmoon Kim <sangmoon.kim@samsung.com>
Change-Id: I00162d88e2a446e9ece4804def098fcdc63fceb9
(cherry picked from commit d497887b00)
This reverts commit 31c9ccb138 (Revert "ANDROID: vendor_hooks: add
waiting information for blocked tasks")
And also revert portions of 396a501b17 (Revert "ANDROID: rwsem: Add
vendor hook to the rw-semaphore")
The original patch has been reverted to resolve merge issues.
This patch adds again the vendor hooks for the original purpose.
Bug: 216016261
Signed-off-by: Sangmoon Kim <sangmoon.kim@samsung.com>
Change-Id: I04ed7b055eee40f7975bd5d74fb73dd080cd76bf
(cherry picked from commit c23da05eac)
Arm Neoverse-N2 and Cortex-A710 cores are affected by an erratum where
the trbe, under some circumstances, might write upto 64bytes to an
address after the Limit as programmed by the TRBLIMITR_EL1.LIMIT.
This might -
- Corrupt a page in the ring buffer, which may corrupt trace from a
previous session, consumed by userspace.
- Hit the guard page at the end of the vmalloc area and raise a fault.
To keep the handling simpler, we always leave the last page from the
range, which TRBE is allowed to write. This can be achieved by ensuring
that we always have more than a PAGE worth space in the range, while
calculating the LIMIT for TRBE. And then the LIMIT pointer can be
adjusted to leave the PAGE (TRBLIMITR.LIMIT -= PAGE_SIZE), out of the
TRBE range while enabling it. This makes sure that the TRBE will only
write to an area within its allowed limit (i.e, [head-head+size]) and
we do not have to handle address faults within the driver.
Bug: 213931796
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Mike Leach <mike.leach@linaro.org>
Cc: Leo Yan <leo.yan@linaro.org>
Cc: Will Deacon <will@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20211019163153.3692640-5-suzuki.poulose@arm.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit 8d81b2a38d)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: Ie4e6439745d33d6c339a5d4ec72d4b3cb5f77f1f
Arm Neoverse-N2 (#2067961) and Cortex-A710 (#2054223) suffers
from errata, where a TSB (trace synchronization barrier)
fails to flush the trace data completely, when executed from
a trace prohibited region. In Linux we always execute it
after we have moved the PE to trace prohibited region. So,
we can apply the workaround every time a TSB is executed.
The work around is to issue two TSB consecutively.
NOTE: This errata is defined as LOCAL_CPU_ERRATUM, implying
that a late CPU could be blocked from booting if it is the
first CPU that requires the workaround. This is because we
do not allow setting a cpu_hwcaps after the SMP boot. The
other alternative is to use "this_cpu_has_cap()" instead
of the faster system wide check, which may be a bit of an
overhead, given we may have to do this in nvhe KVM host
before a guest entry.
Bug: 213931796
Cc: Will Deacon <will@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Mike Leach <mike.leach@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Marc Zyngier <maz@kernel.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20211019163153.3692640-4-suzuki.poulose@arm.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit fa82d0b4b8)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: I293455c7714d91226eb836f422b4f9c41fe16693
Arm Neoverse-N2 and the Cortex-A710 cores are affected
by a CPU erratum where the TRBE will overwrite the trace buffer
in FILL mode. The TRBE doesn't stop (as expected in FILL mode)
when it reaches the limit and wraps to the base to continue
writing upto 3 cache lines. This will overwrite any trace that
was written previously.
Add the Neoverse-N2 erratum(#2139208) and Cortex-A710 erratum
(#2119858) to the detection logic.
This will be used by the TRBE driver in later patches to work
around the issue. The detection has been kept with the core
arm64 errata framework list to make sure :
- We don't duplicate the framework in TRBE driver
- The errata detection is advertised like the rest
of the CPU errata.
Note that the Kconfig entries are not fully active until the
TRBE driver implements the work around.
Bug: 213931796
Cc: Will Deacon <will@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Mike Leach <mike.leach@linaro.org>
cc: Leo Yan <leo.yan@linaro.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20211019163153.3692640-3-suzuki.poulose@arm.com
Signed-off-by: Will Deacon <will@kernel.org>
(cherry picked from commit b9d216fcef)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: Ia46124bc0022f0e7c71b561f95f43462caccd82a
TRBE implementations affected by Arm erratum (2253138 or 2224489), could
write to the next address after the TRBLIMITR.LIMIT, instead of wrapping
to the TRBBASER. This implies that the TRBE could potentially corrupt :
- A page used by the rest of the kernel/user (if the LIMIT = end of
perf ring buffer)
- A page within the ring buffer, but outside the driver's range.
[head, head + size]. This may contain some trace data, may be
consumed by the userspace.
We workaround this erratum by :
- Making sure that there is at least an extra PAGE space left in the
TRBE's range than we normally assign. This will be additional to other
restrictions (e.g, the TRBE alignment for working around
TRBE_WORKAROUND_OVERWRITE_IN_FILL_MODE, where there is a minimum of
PAGE_SIZE. Thus we would have 2 * PAGE_SIZE)
- Adjust the LIMIT to leave the last PAGE_SIZE out of the TRBE's allowed
range (i.e, TRBEBASER...TRBLIMITR.LIMIT), by :
TRBLIMITR.LIMIT -= PAGE_SIZE
Bug: 213931796
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Mike Leach <mike.leach@linaro.org>
Cc: Leo Yan <leo.yan@linaro.org>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20211019163153.3692640-14-suzuki.poulose@arm.com
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit f9efc79d0a)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: I11b66705b6751831c71c219a9d79266afdec7552
ARM Neoverse-N2 (#2139208) and Cortex-A710(##2119858) suffers from
an erratum, which when triggered, might cause the TRBE to overwrite
the trace data already collected in FILL mode, in the event of a WRAP.
i.e, the TRBE doesn't stop writing the data, instead wraps to the base
and could write upto 3 cache line size worth trace. Thus, this could
corrupt the trace at the "BASE" pointer.
The workaround is to program the write pointer 256bytes from the
base, such that if the erratum is triggered, it doesn't overwrite
the trace data that was captured. This skipped region could be
padded with ignore packets at the end of the session, so that
the decoder sees a continuous buffer with some padding at the
beginning. The trace data written at the base is considered
lost as the limit could have been in the middle of the perf
ring buffer, and jumping to the "base" is not acceptable.
We set the flags already to indicate that some amount of trace
was lost during the FILL event IRQ. So this is fine.
One important change with the work around is, we program the
TRBBASER_EL1 to current page where we are allowed to write.
Otherwise, it could overwrite a region that may be consumed
by the perf. Towards this, we always make sure that the
"handle->head" and thus the trbe_write is PAGE_SIZE aligned,
so that we can set the BASE to the PAGE base and move the
TRBPTR to the 256bytes offset.
Bug: 213931796
Cc: Mike Leach <mike.leach@linaro.org>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Leo Yan <leo.yan@linaro.org>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20211019163153.3692640-11-suzuki.poulose@arm.com
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit 5cb75f1880)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: I3db5aaf01114c38fb7a70f43899a196b8d9cbf41
Add a minimal infrastructure to keep track of the errata
affecting the given TRBE instance. Given that we have
heterogeneous CPUs, we have to manage the list per-TRBE
instance to be able to apply the work around as needed.
Thus we will need to check if individual CPUs are affected
by the erratum.
We rely on the arm64 errata framework for the actual
description and the discovery of a given erratum, to
keep the Erratum work around at a central place and
benefit from the code and the advertisement from the
kernel. Though we could reuse the "this_cpu_has_cap()"
to apply an erratum work around, it is a bit of a heavy
operation, as it must go through the "erratum" detection
check on the CPU every time it is called (e.g, scanning
through a table of affected MIDRs). Since we need
to do this check for every session, may be multiple
times (depending on the wrok around), we could save
the cycles by caching the affected errata per-CPU
instance in the per-CPU struct trbe_cpudata.
Since we are only interested in the errata affecting
the TRBE driver, we only need to track a very few of them
per-CPU. Thus we use a local mapping of the CPUCAP for the
erratum to avoid bloating up a bitmap for trbe_cpudata.
i.e, each arm64 TRBE erratum bit is assigned a "index"
within the driver to track. Each trbe instance updates
the list of affected erratum at probe time on the CPU.
This makes sure that we can easily access the list of
errata on a given TRBE instance without much overhead.
Bug: 213931796
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Mike Leach <mike.leach@linaro.org>
Cc: Leo Yan <leo.yan@linaro.org>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20211019163153.3692640-10-suzuki.poulose@arm.com
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit 8a1065127d)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: Ia6907a9626b6f1be6ab86c84b61d241448714028
When the TRBE generates an IRQ, we stop the TRBE, collect the trace
and then reprogram the TRBE with the updated buffer pointers, whenever
possible. We might also leave the TRBE disabled, if there is not
enough space left in the buffer. However, we do not touch the ETE at
all during all of this. This means the ETE is only disabled when
the event is disabled later (via irq_work). This is incorrect, as the
ETE trace is still ON without actually being captured and may be routed
to the ATB (even if it is for a short duration).
So, we move the CPU into trace prohibited state always before disabling
the TRBE, upon entering the IRQ handler. The state is restored if the
TRBE is enabled back. Otherwise the trace remains prohibited.
Since, the ETM/ETE driver now controls the TRFCR_EL1 per session, the
tracing can be restored/enabled back when the event is rescheduled
in.
Bug: 213931796
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Mike Leach <mike.leach@linaro.org>
Cc: Leo Yan <leo.yan@linaro.org>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20210923143919.2944311-6-suzuki.poulose@arm.com
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit dcfecfa444)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: I7cdd787539b94b3a0ab5b9d1f568a1dee396b042
When we detect that there isn't enough space left to start a meaningful
session, we disable the TRBE, marking the buffer as TRUNCATED. But we delay
the notification to the perf layer by perf_aux_output_end() until the event
is scheduled out, triggered from the kernel perf layer. This will cause
significant black outs in the trace. Now that the CoreSight PMU layer can
handle a closed "AUX" handle properly, we can close the handle as soon as
we detect the case, allowing the userspace to collect and re-enable the
event.
Also, while in the IRQ handler, move the irq_work_run() after we have
updated the handle, to make sure the "TRUNCATED" flag causes the event to
be disabled as soon as possible.
Bug: 213931796
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Mike Leach <mike.leach@linaro.org>
Cc: Leo Yan <leo.yan@linaro.org>
Cc: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Will Deacon <will@kernel.org>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20210923143919.2944311-5-suzuki.poulose@arm.com
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit 9bef9d0850)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: I3a29de03d1124e0576304b4b64a36585d9ecc422
The TRBE driver marks the AUX buffer as TRUNCATED when we get an IRQ
on FILL event. This has rather unwanted side-effect of the event
being disabled when there may be more space in the ring buffer.
So, instead of TRUNCATE we need a different flag to indicate
that the trace may have lost a few bytes (i.e from the point of
generating the FILL event until the IRQ is consumed). Anyways, the
userspace must use the size from RECORD_AUX headers to restrict
the "trace" decoding.
Using PARTIAL flag causes the perf tool to generate the
following warning:
Warning:
AUX data had gaps in it XX times out of YY!
Are you running a KVM guest in the background?
which is pointlessly scary for a user. The other remaining options
are :
- COLLISION - Use by SPE to indicate samples collided
- Add a new flag - Specifically for CoreSight, doesn't sound
so good, if we can re-use something.
Given that we don't already use the "COLLISION" flag, the above
behavior can be notified using this flag for CoreSight.
Bug: 213931796
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: James Clark <james.clark@arm.com>
Cc: Mike Leach <mike.leach@linaro.org>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Leo Yan <leo.yan@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20210923143919.2944311-4-suzuki.poulose@arm.com
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit 0a5f355633)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: I56b2eaa6b171bea000c040d94640d59fbfa3cb27
On a spurious IRQ, right now we disable the TRBE and then re-enable
it back, resetting the "buffer" pointers(i.e BASE, LIMIT and more
importantly WRITE) to the original pointers from the AUX handle.
This implies that we overwrite any trace that was written so far,
(by overwriting TRBPTR) while we should have ignored the IRQ.
On detecting a spurious IRQ after examining the TRBSR we simply
re-enable the TRBE without touching the other parameters.
Bug: 213931796
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Mike Leach <mike.leach@linaro.org>
Cc: Leo Yan <leo.yan@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20210923143919.2944311-3-suzuki.poulose@arm.com
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit 7037a39d37)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: I83d52e150ecafa8830cf0d1be0f870958b3f432a
We mark the buffer as TRUNCATED when there is no space left
in the buffer. But we do it at different points.
__trbe_normal_offset()
and also, at all the callers of the above function via
compute_trbe_buffer_limit(), when the limit == base (i.e
offset = 0 as returned by the __trbe_normal_offset()).
So, given that the callers already mark the buffer as TRUNCATED
drop the caller inside the __trbe_normal_offset().
This is in preparation to moving the handling of TRUNCATED
into a central place.
Bug: 213931796
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Mike Leach <mike.leach@linaro.org>
Cc: Leo Yan <leo.yan@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Link: https://lore.kernel.org/r/20210914102641.1852544-6-suzuki.poulose@arm.com
[Moved comment as Anshuman requested]
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit acee3ef86d)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: I94ed5a532138c12dfc84710f1884b24fbbbaf34c
When the TRBE is stopped on truncating an event, we may not
set the FORMAT flag, even though the size of the record is 0.
Let us be consistent and not confuse the user.
To ensure that the format flag is always set on all the
records generated by TRBE, set the flag when we have a
new handle. Rather than deferring to the "end" operation,
which makes it clear. So, we can do this from
- arm_trbe_enable() -> When a new handle is provided by the
CoreSight PMU, triggered via etm_event_start()
- trbe_handle_overflow() -> When we begin a new handle after
closing the previous on overflow.
Bug: 213931796
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Leo Yan <leo.yan@linaro.org>
Cc: Mike Leach <mike.leach@linaro.org>
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Link: https://lore.kernel.org/r/20210914102641.1852544-5-suzuki.poulose@arm.com
[Fixed inverted words in title]
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit 5bd9ff830c)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: Ia97b517aff176c88279fcf09619f6ac1ed831bf9
The Trace Filtering support (FEAT_TRF) ensures that the ETM
can be prohibited from generating any trace for a given EL.
This is much stricter knob, than the TRCVICTLR exception level
masks, which doesn't prevent the ETM from generating Context
packets for an "excluded" EL. At the moment, we do a onetime
enable trace at user and kernel and leave it untouched for the
kernel life time. This implies that the ETM could potentially
generate trace packets containing the kernel addresses, and
thus leaking the kernel virtual address in the trace.
This patch makes the switch dynamic, by honoring the filters
set by the user and enforcing them in the TRFCR controls.
We also rename the cpu_enable_tracing() appropriately to
cpu_detect_trace_filtering() and the drvdata member
trfc => trfcr to indicate the "value" of the TRFCR_EL1.
Bug: 213931796
Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Cc: Al Grant <al.grant@arm.com>
Cc: Mike Leach <mike.leach@linaro.org>
Cc: Leo Yan <leo.yan@linaro.org>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Link: https://lore.kernel.org/r/20210914102641.1852544-3-suzuki.poulose@arm.com
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit 5f6fd1aa8c)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: I4b0653306cad6f3a52cb6ab02e1dffe12d9c3fef
When a traced process runs on a CPU that can't reach the selected sink,
the event will be stopped with PERF_HES_STOPPED. This means that even if
the process migrates to a valid CPU, tracing will not resume.
This can be reproduced (on N1SDP) by using taskset to start the process
on CPU 0, and then switching it to CPU 2 (ETF 1 is only reachable from
CPU 2):
taskset --cpu-list 0 ./perf record -e cs_etm/@tmc_etf1/ --per-thread -- taskset --cpu-list 2 ls
This produces a single 0 length AUX record, and then no more trace:
0x3c8 [0x30]: PERF_RECORD_AUX offset: 0 size: 0 flags: 0x1 [T]
After the fix, the same command produces normal AUX records. The perf
self test "89: Check Arm CoreSight trace data recording and synthesized
samples" no longer fails intermittently. This was because the taskset in
the test is after the fork, so there is a period where the task is
scheduled on a random CPU rather than forced to a valid one.
Specifically selecting an invalid CPU will still result in a failure to
open the event because it will never produce trace:
./perf record -C 2 -e cs_etm/@tmc_etf0/
failed to mmap with 12 (Cannot allocate memory)
The only scenario that has changed is if the CPU mask has a valid CPU
sink combo in it.
Testing
=======
* Coresight self test passes consistently:
./perf test Coresight
* CPU wide mode still produces trace:
./perf record -e cs_etm// -a
* Invalid -C options still fail to open:
./perf record -C 2,3 -e cs_etm/@tmc_etf0/
failed to mmap with 12 (Cannot allocate memory)
* Migrating a task to a valid sink/CPU now produces trace:
taskset --cpu-list 0 ./perf record -e cs_etm/@tmc_etf1/ --per-thread -- taskset --cpu-list 2 ls
* If the task remains on an invalid CPU, no trace is emitted:
taskset --cpu-list 0 ./perf record -e cs_etm/@tmc_etf1/ --per-thread -- ls
Bug: 213931796
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: James Clark <james.clark@arm.com>
Link: https://lore.kernel.org/r/20210922125144.133872-2-james.clark@arm.com
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit 8c60acbcb9)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: If0d5639287630bb4ceb6defcebed15d139a1a8b2
The AUX bounce buffer is allocated with API dma_alloc_coherent(), in the
low level's architecture code, e.g. for Arm64, it maps the memory with
the attribution "Normal non-cacheable"; this can be concluded from the
definition for pgprot_dmacoherent() in arch/arm64/include/asm/pgtable.h.
Later when access the AUX bounce buffer, since the memory mapping is
non-cacheable, it's low efficiency due to every load instruction must
reach out DRAM.
This patch changes to allocate pages with dma_alloc_noncoherent(), the
driver can access the memory via cacheable mapping; therefore, load
instructions can fetch data from cache lines rather than always read
data from DRAM, the driver can boost memory performance. After using
the cacheable mapping, the driver uses dma_sync_single_for_cpu() to
invalidate cacheline prior to read bounce buffer so can avoid read stale
trace data.
By measurement the duration for function tmc_update_etr_buffer() with
ftrace function_graph tracer, it shows the performance significant
improvement for copying 4MiB data from bounce buffer:
# echo tmc_etr_get_data_flat_buf > set_graph_notrace // avoid noise
# echo tmc_update_etr_buffer > set_graph_function
# echo function_graph > current_tracer
before:
# CPU DURATION FUNCTION CALLS
# | | | | | | |
2) | tmc_update_etr_buffer() {
...
2) # 8148.320 us | }
after:
# CPU DURATION FUNCTION CALLS
# | | | | | | |
2) | tmc_update_etr_buffer() {
...
2) # 2525.420 us | }
Bug: 213931796
Signed-off-by: Leo Yan <leo.yan@linaro.org>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20210905032144.966766-1-leo.yan@linaro.org
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit 0abd076217)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: I6fb153777076db8abeaa5145cb6050ddfed35781
Commit 2f01c200d4 ("perf cs-etm: Remove callback cs_etm_find_snapshot()")
has removed the function cs_etm_find_snapshot() from the perf tool in the
user space, now CoreSight trace directly uses the perf common function
__auxtrace_mmap__read() to calcualte the head and size for AUX trace data
in snapshot mode.
This patch updates the comments in drivers to make them generic and not
stick to any specific function from perf tool.
Bug: 213931796
Signed-off-by: Leo Yan <leo.yan@linaro.org>
Link: https://lore.kernel.org/r/20210912125748.2816606-3-leo.yan@linaro.org
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit 7ba7ae1d5a)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: I82e4ac325242429cfdc3da67e312c7b2e0cd8f3e
When enable the Arm CoreSight PMU event, the context for AUX ring buffer
is prepared in the structure perf_output_handle, and its field "head"
points the head of the AUX ring buffer and it is updated after filling
AUX trace data into buffer.
Current code uses an extra field etr_perf_buffer::head to maintain the
header for the AUX ring buffer which is not necessary; alternatively,
it's better to directly use perf_output_handle::head.
This patch removes the field etr_perf_buffer::head and directly uses
perf_output_handle::head for the head of AUX ring buffer.
Bug: 213931796
Signed-off-by: Leo Yan <leo.yan@linaro.org>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20210912125748.2816606-2-leo.yan@linaro.org
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit f36dec8da1)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: Ib5fee976b5092936da029bee419ccc6b02e9407c
The current driver sets the write burst size initiated by TMC-ETR on
AXI bus to a fixed value of 16. Make this configurable by reading the
value specified in fwnode. If not specified, then default to 16.
Introduced a "max_burst_size" variable in tmc_drvdata structure to
facilitate this change.
Bug: 213931796
Signed-off-by: Tanmay Jagdale <tanmay@marvell.com>
Reviewed-by: Mike Leach <mike.leach@linaro.org>
Link: https://lore.kernel.org/r/20210901131049.1365367-3-tanmay@marvell.com
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit 4d5d88baa6)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: I9ad7306daa557e5afa4e9680485bff60fe7ec239
Debugfs is nice and so are module parameters, but
* debugfs doesn't take effect early (e.g., if drivers are locking up
before user space gets anywhere)
* module parameters either add a lot to the kernel command line, or
else take effect late as well (if you build =m and configure in
/etc/modprobe.d/)
So in the same spirit as these
CONFIG_PANIC_ON_OOPS (also available via cmdline or modparam)
CONFIG_INTEL_IOMMU_DEFAULT_ON (also available via cmdline)
add a new Kconfig option.
Module parameters and debugfs can still override.
Bug: 213931796
Signed-off-by: Brian Norris <briannorris@chromium.org>
Reviewed-by: Leo Yan <leo.yan@linaro.org>
[Fixed missing double quote in Kconfig title]
Link: https://lore.kernel.org/r/20210903182839.1.I20856983f2841b78936134dcf9cdf6ecafe632b9@changeid
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
(cherry picked from commit 204879e699)
Signed-off-by: Qais Yousef <qais.yousef@arm.com>
Change-Id: I135172e06c96701d7ed0d928d92974fd85039243
Allow module to control behavior of get_nohz_timer_target.
Bug: 205164003
Change-Id: I38cb201ebf06db7bbce0d6cb68dbbe3729355be8
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
