shinys000114/linux
1
0
Fork 0
mirror of https://github.com/hardkernel/linux.git synced 2026-06-06 10:58:48 +09:00
Code Issues Packages Projects Releases Wiki Activity

Merge c5c009e250 ("Merge tag 'slab-for-5.18' of git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab") into android-mainline

Browse Source 
Steps on the way to 5.18-rc1

Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: I69954395df00a8ee8247c9edef82d9de5e9c8129
This commit is contained in:
Greg Kroah-Hartman
2022-04-03 12:28:57 +02:00
parent 1b523b5b76 c5c009e250
commit b2c024c2ba
61 changed files with 5289 additions and 717 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
Documentation/admin-guide/kernel-parameters.txt
View File

@@ -724,6 +724,12 @@
hvc<n> Use the hypervisor console device <n>. This is for
both Xen and PowerPC hypervisors.
{ null | "" }
Use to disable console output, i.e., to have kernel
console messages discarded.
This must be the only console= parameter used on the
kernel command line.
If the device connected to the port is not a TTY but a braille
device, prepend "brl," before the device type, for instance
console=brl,ttyS0
@@ -1459,6 +1465,14 @@
as early as possible in order to facilitate early
boot debugging.
ftrace_boot_snapshot
[FTRACE] On boot up, a snapshot will be taken of the
ftrace ring buffer that can be read at:
/sys/kernel/tracing/snapshot.
This is useful if you need tracing information from kernel
boot up that is likely to be overridden by user space
start up functionality.
ftrace_dump_on_oops[=orig_cpu]
[FTRACE] will dump the trace buffers on oops.
If no parameter is passed, ftrace will dump
@@ -3520,8 +3534,7 @@
difficult since unequal pointers can no longer be
compared. However, if this command-line option is
specified, then all normal pointers will have their true
value printed. Pointers printed via %pK may still be
hashed. This option should only be specified when
value printed. This option should only be specified when
debugging the kernel. Please do not use on production
kernels.

19
Documentation/tools/rtla/common_options.rst
View File

@@ -14,6 +14,25 @@
Save the stopped trace to [*file|osnoise_trace.txt*].
**-e**, **--event** *sys:event*
Enable an event in the trace (**-t**) session. The argument can be a specific event, e.g., **-e** *sched:sched_switch*, or all events of a system group, e.g., **-e** *sched*. Multiple **-e** are allowed. It is only active when **-t** or **-a** are set.
**--filter** *<filter>*
Filter the previous **-e** *sys:event* event with *<filter>*. For further information about event filtering see https://www.kernel.org/doc/html/latest/trace/events.html#event-filtering.
**--trigger** *<trigger>*
Enable a trace event trigger to the previous **-e** *sys:event*.
If the *hist:* trigger is activated, the output histogram will be automatically saved to a file named *system_event_hist.txt*.
For example, the command:
rtla <command> <mode> -t -e osnoise:irq_noise --trigger="hist:key=desc,duration/1000:sort=desc,duration/1000:vals=hitcount"
Will automatically save the content of the histogram associated to *osnoise:irq_noise* event in *osnoise_irq_noise_hist.txt*.
For further information about event trigger see https://www.kernel.org/doc/html/latest/trace/events.html#event-triggers.
**-P**, **--priority** *o:prio|r:prio|f:prio|d:runtime:period*
Set scheduling parameters to the osnoise tracer threads, the format to set the priority are:

10
Documentation/tools/rtla/common_osnoise_options.rst
View File

@@ -1,3 +1,8 @@
**-a**, **--auto** *us*
Set the automatic trace mode. This mode sets some commonly used options
while debugging the system. It is equivalent to use **-s** *us* **-T 1 -t**.
**-p**, **--period** *us*
Set the *osnoise* tracer period in microseconds.
@@ -15,3 +20,8 @@
Stop the trace if the total sample is higher than the argument in microseconds.
If **-T** is set, it will also save the trace to the output.
**-T**, **--threshold** *us*
Specify the minimum delta between two time reads to be considered noise.
The default threshold is *5 us*.

12
Documentation/tools/rtla/common_timerlat_options.rst
View File

@@ -1,3 +1,10 @@
**-a**, **--auto** *us*
Set the automatic trace mode. This mode sets some commonly used options
while debugging the system. It is equivalent to use **-T** *us* **-s** *us*
**-t**. By default, *timerlat* tracer uses FIFO:95 for *timerlat* threads,
thus equilavent to **-P** *f:95*.
**-p**, **--period** *us*
Set the *timerlat* tracer period in microseconds.
@@ -14,3 +21,8 @@
Save the stack trace at the *IRQ* if a *Thread* latency is higher than the
argument in us.
**--dma-latency** *us*
Set the /dev/cpu_dma_latency to *us*, aiming to bound exit from idle latencies.
*cyclictest* sets this value to *0* by default, use **--dma-latency** *0* to have
similar results.

1
Documentation/trace/index.rst
View File

@@ -30,3 +30,4 @@ Linux Tracing Technologies
stm
sys-t
coresight/index
user_events

216
Documentation/trace/user_events.rst Normal file
View File

@@ -0,0 +1,216 @@
=========================================
user_events: User-based Event Tracing
=========================================
:Author: Beau Belgrave
Overview
--------
User based trace events allow user processes to create events and trace data
that can be viewed via existing tools, such as ftrace, perf and eBPF.
To enable this feature, build your kernel with CONFIG_USER_EVENTS=y.
Programs can view status of the events via
/sys/kernel/debug/tracing/user_events_status and can both register and write
data out via /sys/kernel/debug/tracing/user_events_data.
Programs can also use /sys/kernel/debug/tracing/dynamic_events to register and
delete user based events via the u: prefix. The format of the command to
dynamic_events is the same as the ioctl with the u: prefix applied.
Typically programs will register a set of events that they wish to expose to
tools that can read trace_events (such as ftrace and perf). The registration
process gives back two ints to the program for each event. The first int is the
status index. This index describes which byte in the
/sys/kernel/debug/tracing/user_events_status file represents this event. The
second int is the write index. This index describes the data when a write() or
writev() is called on the /sys/kernel/debug/tracing/user_events_data file.
The structures referenced in this document are contained with the
/include/uap/linux/user_events.h file in the source tree.
**NOTE:** *Both user_events_status and user_events_data are under the tracefs
filesystem and may be mounted at different paths than above.*
Registering
-----------
Registering within a user process is done via ioctl() out to the
/sys/kernel/debug/tracing/user_events_data file. The command to issue is
DIAG_IOCSREG.
This command takes a struct user_reg as an argument::
struct user_reg {
u32 size;
u64 name_args;
u32 status_index;
u32 write_index;
};
The struct user_reg requires two inputs, the first is the size of the structure
to ensure forward and backward compatibility. The second is the command string
to issue for registering. Upon success two outputs are set, the status index
and the write index.
User based events show up under tracefs like any other event under the
subsystem named "user_events". This means tools that wish to attach to the
events need to use /sys/kernel/debug/tracing/events/user_events/[name]/enable
or perf record -e user_events:[name] when attaching/recording.
**NOTE:** *The write_index returned is only valid for the FD that was used*
Command Format
^^^^^^^^^^^^^^
The command string format is as follows::
name[:FLAG1[,FLAG2...]] [Field1[;Field2...]]
Supported Flags
^^^^^^^^^^^^^^^
**BPF_ITER** - EBPF programs attached to this event will get the raw iovec
struct instead of any data copies for max performance.
Field Format
^^^^^^^^^^^^
::
type name [size]
Basic types are supported (__data_loc, u32, u64, int, char, char[20], etc).
User programs are encouraged to use clearly sized types like u32.
**NOTE:** *Long is not supported since size can vary between user and kernel.*
The size is only valid for types that start with a struct prefix.
This allows user programs to describe custom structs out to tools, if required.
For example, a struct in C that looks like this::
struct mytype {
char data[20];
};
Would be represented by the following field::
struct mytype myname 20
Deleting
-----------
Deleting an event from within a user process is done via ioctl() out to the
/sys/kernel/debug/tracing/user_events_data file. The command to issue is
DIAG_IOCSDEL.
This command only requires a single string specifying the event to delete by
its name. Delete will only succeed if there are no references left to the
event (in both user and kernel space). User programs should use a separate file
to request deletes than the one used for registration due to this.
Status
------
When tools attach/record user based events the status of the event is updated
in realtime. This allows user programs to only incur the cost of the write() or
writev() calls when something is actively attached to the event.
User programs call mmap() on /sys/kernel/debug/tracing/user_events_status to
check the status for each event that is registered. The byte to check in the
file is given back after the register ioctl() via user_reg.status_index.
Currently the size of user_events_status is a single page, however, custom
kernel configurations can change this size to allow more user based events. In
all cases the size of the file is a multiple of a page size.
For example, if the register ioctl() gives back a status_index of 3 you would
check byte 3 of the returned mmap data to see if anything is attached to that
event.
Administrators can easily check the status of all registered events by reading
the user_events_status file directly via a terminal. The output is as follows::
Byte:Name [# Comments]
...
Active: ActiveCount
Busy: BusyCount
Max: MaxCount
For example, on a system that has a single event the output looks like this::
1:test
Active: 1
Busy: 0
Max: 4096
If a user enables the user event via ftrace, the output would change to this::
1:test # Used by ftrace
Active: 1
Busy: 1
Max: 4096
**NOTE:** *A status index of 0 will never be returned. This allows user
programs to have an index that can be used on error cases.*
Status Bits
^^^^^^^^^^^
The byte being checked will be non-zero if anything is attached. Programs can
check specific bits in the byte to see what mechanism has been attached.
The following values are defined to aid in checking what has been attached:
**EVENT_STATUS_FTRACE** - Bit set if ftrace has been attached (Bit 0).
**EVENT_STATUS_PERF** - Bit set if perf/eBPF has been attached (Bit 1).
Writing Data
------------
After registering an event the same fd that was used to register can be used
to write an entry for that event. The write_index returned must be at the start
of the data, then the remaining data is treated as the payload of the event.
For example, if write_index returned was 1 and I wanted to write out an int
payload of the event. Then the data would have to be 8 bytes (2 ints) in size,
with the first 4 bytes being equal to 1 and the last 4 bytes being equal to the
value I want as the payload.
In memory this would look like this::
int index;
int payload;
User programs might have well known structs that they wish to use to emit out
as payloads. In those cases writev() can be used, with the first vector being
the index and the following vector(s) being the actual event payload.
For example, if I have a struct like this::
struct payload {
int src;
int dst;
int flags;
};
It's advised for user programs to do the following::
struct iovec io[2];
struct payload e;
io[0].iov_base = &write_index;
io[0].iov_len = sizeof(write_index);
io[1].iov_base = &e;
io[1].iov_len = sizeof(e);
writev(fd, (const struct iovec*)io, 2);
**NOTE:** *The write_index is not emitted out into the trace being recorded.*
EBPF
----
EBPF programs that attach to a user-based event tracepoint are given a pointer
to a struct user_bpf_context. The bpf context contains the data type (which can
be a user or kernel buffer, or can be a pointer to the iovec) and the data
length that was emitted (minus the write_index).
Example Code
------------
See sample code in samples/user_events.

2
include/linux/cacheflush.h
View File

@@ -4,6 +4,8 @@
#include <asm/cacheflush.h>
struct folio;
#if ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE
#ifndef ARCH_IMPLEMENTS_FLUSH_DCACHE_FOLIO
void flush_dcache_folio(struct folio *folio);

11
include/linux/ftrace.h
View File

@@ -30,6 +30,12 @@
#define ARCH_SUPPORTS_FTRACE_OPS 0
#endif
#ifdef CONFIG_TRACING
extern void ftrace_boot_snapshot(void);
#else
static inline void ftrace_boot_snapshot(void) { }
#endif
#ifdef CONFIG_FUNCTION_TRACER
struct ftrace_ops;
struct ftrace_regs;
@@ -215,7 +221,10 @@ struct ftrace_ops_hash {
void ftrace_free_init_mem(void);
void ftrace_free_mem(struct module *mod, void *start, void *end);
#else
static inline void ftrace_free_init_mem(void) { }
static inline void ftrace_free_init_mem(void)
{
ftrace_boot_snapshot();
}
static inline void ftrace_free_mem(struct module *mod, void *start, void *end) { }
#endif

3
include/linux/slab.h
View File

@@ -117,9 +117,6 @@
#define SLAB_RECLAIM_ACCOUNT ((slab_flags_t __force)0x00020000U)
#define SLAB_TEMPORARY SLAB_RECLAIM_ACCOUNT /* Objects are short-lived */
/* Slab deactivation flag */
#define SLAB_DEACTIVATED ((slab_flags_t __force)0x10000000U)
/*
* ZERO_SIZE_PTR will be returned for zero sized kmalloc requests.
*

24
include/linux/trace_events.h
View File

@@ -315,6 +315,7 @@ enum {
TRACE_EVENT_FL_KPROBE_BIT,
TRACE_EVENT_FL_UPROBE_BIT,
TRACE_EVENT_FL_EPROBE_BIT,
TRACE_EVENT_FL_CUSTOM_BIT,
};
/*
@@ -328,6 +329,9 @@ enum {
* KPROBE - Event is a kprobe
* UPROBE - Event is a uprobe
* EPROBE - Event is an event probe
* CUSTOM - Event is a custom event (to be attached to an exsiting tracepoint)
* This is set when the custom event has not been attached
* to a tracepoint yet, then it is cleared when it is.
*/
enum {
TRACE_EVENT_FL_FILTERED = (1 << TRACE_EVENT_FL_FILTERED_BIT),
@@ -339,6 +343,7 @@ enum {
TRACE_EVENT_FL_KPROBE = (1 << TRACE_EVENT_FL_KPROBE_BIT),
TRACE_EVENT_FL_UPROBE = (1 << TRACE_EVENT_FL_UPROBE_BIT),
TRACE_EVENT_FL_EPROBE = (1 << TRACE_EVENT_FL_EPROBE_BIT),
TRACE_EVENT_FL_CUSTOM = (1 << TRACE_EVENT_FL_CUSTOM_BIT),
};
#define TRACE_EVENT_FL_UKPROBE (TRACE_EVENT_FL_KPROBE | TRACE_EVENT_FL_UPROBE)
@@ -440,7 +445,9 @@ static inline bool bpf_prog_array_valid(struct trace_event_call *call)
static inline const char *
trace_event_name(struct trace_event_call *call)
{
if (call->flags & TRACE_EVENT_FL_TRACEPOINT)
if (call->flags & TRACE_EVENT_FL_CUSTOM)
return call->name;
else if (call->flags & TRACE_EVENT_FL_TRACEPOINT)
return call->tp ? call->tp->name : NULL;
else
return call->name;
@@ -903,3 +910,18 @@ perf_trace_buf_submit(void *raw_data, int size, int rctx, u16 type,
#endif
#endif /* _LINUX_TRACE_EVENT_H */
/*
* Note: we keep the TRACE_CUSTOM_EVENT outside the include file ifdef protection.
* This is due to the way trace custom events work. If a file includes two
* trace event headers under one "CREATE_CUSTOM_TRACE_EVENTS" the first include
* will override the TRACE_CUSTOM_EVENT and break the second include.
*/
#ifndef TRACE_CUSTOM_EVENT
#define DECLARE_CUSTOM_EVENT_CLASS(name, proto, args, tstruct, assign, print)
#define DEFINE_CUSTOM_EVENT(template, name, proto, args)
#define TRACE_CUSTOM_EVENT(name, proto, args, struct, assign, print)
#endif /* ifdef TRACE_CUSTOM_EVENT (see note above) */

77
include/trace/define_custom_trace.h Normal file
View File

@@ -0,0 +1,77 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Trace files that want to automate creation of all tracepoints defined
* in their file should include this file. The following are macros that the
* trace file may define:
*
* TRACE_SYSTEM defines the system the tracepoint is for
*
* TRACE_INCLUDE_FILE if the file name is something other than TRACE_SYSTEM.h
* This macro may be defined to tell define_trace.h what file to include.
* Note, leave off the ".h".
*
* TRACE_INCLUDE_PATH if the path is something other than core kernel include/trace
* then this macro can define the path to use. Note, the path is relative to
* define_trace.h, not the file including it. Full path names for out of tree
* modules must be used.
*/
#ifdef CREATE_CUSTOM_TRACE_EVENTS
/* Prevent recursion */
#undef CREATE_CUSTOM_TRACE_EVENTS
#include <linux/stringify.h>
#undef TRACE_CUSTOM_EVENT
#define TRACE_CUSTOM_EVENT(name, proto, args, tstruct, assign, print)
#undef DEFINE_CUSTOM_EVENT
#define DEFINE_CUSTOM_EVENT(template, name, proto, args)
#undef TRACE_INCLUDE
#undef __TRACE_INCLUDE
#ifndef TRACE_INCLUDE_FILE
# define TRACE_INCLUDE_FILE TRACE_SYSTEM
# define UNDEF_TRACE_INCLUDE_FILE
#endif
#ifndef TRACE_INCLUDE_PATH
# define __TRACE_INCLUDE(system) <trace/events/system.h>
# define UNDEF_TRACE_INCLUDE_PATH
#else
# define __TRACE_INCLUDE(system) __stringify(TRACE_INCLUDE_PATH/system.h)
#endif
# define TRACE_INCLUDE(system) __TRACE_INCLUDE(system)
/* Let the trace headers be reread */
#define TRACE_CUSTOM_MULTI_READ
#include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
#ifdef TRACEPOINTS_ENABLED
#include <trace/trace_custom_events.h>
#endif
#undef TRACE_CUSTOM_EVENT
#undef DECLARE_CUSTOM_EVENT_CLASS
#undef DEFINE_CUSTOM_EVENT
#undef TRACE_CUSTOM_MULTI_READ
/* Only undef what we defined in this file */
#ifdef UNDEF_TRACE_INCLUDE_FILE
# undef TRACE_INCLUDE_FILE
# undef UNDEF_TRACE_INCLUDE_FILE
#endif
#ifdef UNDEF_TRACE_INCLUDE_PATH
# undef TRACE_INCLUDE_PATH
# undef UNDEF_TRACE_INCLUDE_PATH
#endif
/* We may be processing more files */
#define CREATE_CUSTOM_TRACE_POINTS
#endif /* CREATE_CUSTOM_TRACE_POINTS */

37
include/trace/stages/init.h Normal file
View File

@@ -0,0 +1,37 @@
#define __app__(x, y) str__##x##y
#define __app(x, y) __app__(x, y)
#define TRACE_SYSTEM_STRING __app(TRACE_SYSTEM_VAR,__trace_system_name)
#define TRACE_MAKE_SYSTEM_STR() \
static const char TRACE_SYSTEM_STRING[] = \
__stringify(TRACE_SYSTEM)
TRACE_MAKE_SYSTEM_STR();
#undef TRACE_DEFINE_ENUM
#define TRACE_DEFINE_ENUM(a) \
static struct trace_eval_map __used __initdata \
__##TRACE_SYSTEM##_##a = \
{ \
.system = TRACE_SYSTEM_STRING, \
.eval_string = #a, \
.eval_value = a \
}; \
static struct trace_eval_map __used \
__section("_ftrace_eval_map") \
*TRACE_SYSTEM##_##a = &__##TRACE_SYSTEM##_##a
#undef TRACE_DEFINE_SIZEOF
#define TRACE_DEFINE_SIZEOF(a) \
static struct trace_eval_map __used __initdata \
__##TRACE_SYSTEM##_##a = \
{ \
.system = TRACE_SYSTEM_STRING, \
.eval_string = "sizeof(" #a ")", \
.eval_value = sizeof(a) \
}; \
static struct trace_eval_map __used \
__section("_ftrace_eval_map") \
*TRACE_SYSTEM##_##a = &__##TRACE_SYSTEM##_##a

51
include/trace/stages/stage1_defines.h Normal file
View File

@@ -0,0 +1,51 @@
/* SPDX-License-Identifier: GPL-2.0 */
/* Stage 1 definitions for creating trace events */
#undef __field
#define __field(type, item) type item;
#undef __field_ext
#define __field_ext(type, item, filter_type) type item;
#undef __field_struct
#define __field_struct(type, item) type item;
#undef __field_struct_ext
#define __field_struct_ext(type, item, filter_type) type item;
#undef __array
#define __array(type, item, len) type item[len];
#undef __dynamic_array
#define __dynamic_array(type, item, len) u32 __data_loc_##item;
#undef __string
#define __string(item, src) __dynamic_array(char, item, -1)
#undef __string_len
#define __string_len(item, src, len) __dynamic_array(char, item, -1)
#undef __bitmask
#define __bitmask(item, nr_bits) __dynamic_array(char, item, -1)
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#undef __rel_dynamic_array
#define __rel_dynamic_array(type, item, len) u32 __rel_loc_##item;
#undef __rel_string
#define __rel_string(item, src) __rel_dynamic_array(char, item, -1)
#undef __rel_string_len
#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, -1)
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(char, item, -1)
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)
#undef TP_STRUCT__entry
#define TP_STRUCT__entry(args...) args

54
include/trace/stages/stage2_defines.h Normal file
View File

@@ -0,0 +1,54 @@
/* SPDX-License-Identifier: GPL-2.0 */
/* Stage 2 definitions for creating trace events */
#undef TRACE_DEFINE_ENUM
#define TRACE_DEFINE_ENUM(a)
#undef TRACE_DEFINE_SIZEOF
#define TRACE_DEFINE_SIZEOF(a)
#undef __field
#define __field(type, item)
#undef __field_ext
#define __field_ext(type, item, filter_type)
#undef __field_struct
#define __field_struct(type, item)
#undef __field_struct_ext
#define __field_struct_ext(type, item, filter_type)
#undef __array
#define __array(type, item, len)
#undef __dynamic_array
#define __dynamic_array(type, item, len) u32 item;
#undef __string
#define __string(item, src) __dynamic_array(char, item, -1)
#undef __string_len
#define __string_len(item, src, len) __dynamic_array(char, item, -1)
#undef __bitmask
#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#undef __rel_dynamic_array
#define __rel_dynamic_array(type, item, len) u32 item;
#undef __rel_string
#define __rel_string(item, src) __rel_dynamic_array(char, item, -1)
#undef __rel_string_len
#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, -1)
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, -1)
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)

135
include/trace/stages/stage3_defines.h Normal file
View File

@@ -0,0 +1,135 @@
/* SPDX-License-Identifier: GPL-2.0 */
/* Stage 3 definitions for creating trace events */
#undef __entry
#define __entry field
#undef TP_printk
#define TP_printk(fmt, args...) fmt "\n", args
#undef __get_dynamic_array
#define __get_dynamic_array(field) \
((void *)__entry + (__entry->__data_loc_##field & 0xffff))
#undef __get_dynamic_array_len
#define __get_dynamic_array_len(field) \
((__entry->__data_loc_##field >> 16) & 0xffff)
#undef __get_str
#define __get_str(field) ((char *)__get_dynamic_array(field))
#undef __get_rel_dynamic_array
#define __get_rel_dynamic_array(field) \
((void *)__entry + \
offsetof(typeof(*__entry), __rel_loc_##field) + \
sizeof(__entry->__rel_loc_##field) + \
(__entry->__rel_loc_##field & 0xffff))
#undef __get_rel_dynamic_array_len
#define __get_rel_dynamic_array_len(field) \
((__entry->__rel_loc_##field >> 16) & 0xffff)
#undef __get_rel_str
#define __get_rel_str(field) ((char *)__get_rel_dynamic_array(field))
#undef __get_bitmask
#define __get_bitmask(field) \
({ \
void *__bitmask = __get_dynamic_array(field); \
unsigned int __bitmask_size; \
__bitmask_size = __get_dynamic_array_len(field); \
trace_print_bitmask_seq(p, __bitmask, __bitmask_size); \
})
#undef __get_rel_bitmask
#define __get_rel_bitmask(field) \
({ \
void *__bitmask = __get_rel_dynamic_array(field); \
unsigned int __bitmask_size; \
__bitmask_size = __get_rel_dynamic_array_len(field); \
trace_print_bitmask_seq(p, __bitmask, __bitmask_size); \
})
#undef __get_sockaddr
#define __get_sockaddr(field) ((struct sockaddr *)__get_dynamic_array(field))
#undef __get_rel_sockaddr
#define __get_rel_sockaddr(field) ((struct sockaddr *)__get_rel_dynamic_array(field))
#undef __print_flags
#define __print_flags(flag, delim, flag_array...) \
({ \
static const struct trace_print_flags __flags[] = \
{ flag_array, { -1, NULL }}; \
trace_print_flags_seq(p, delim, flag, __flags); \
})
#undef __print_symbolic
#define __print_symbolic(value, symbol_array...) \
({ \
static const struct trace_print_flags symbols[] = \
{ symbol_array, { -1, NULL }}; \
trace_print_symbols_seq(p, value, symbols); \
})
#undef __print_flags_u64
#undef __print_symbolic_u64
#if BITS_PER_LONG == 32
#define __print_flags_u64(flag, delim, flag_array...) \
({ \
static const struct trace_print_flags_u64 __flags[] = \
{ flag_array, { -1, NULL } }; \
trace_print_flags_seq_u64(p, delim, flag, __flags); \
})
#define __print_symbolic_u64(value, symbol_array...) \
({ \
static const struct trace_print_flags_u64 symbols[] = \
{ symbol_array, { -1, NULL } }; \
trace_print_symbols_seq_u64(p, value, symbols); \
})
#else
#define __print_flags_u64(flag, delim, flag_array...) \
__print_flags(flag, delim, flag_array)
#define __print_symbolic_u64(value, symbol_array...) \
__print_symbolic(value, symbol_array)
#endif
#undef __print_hex
#define __print_hex(buf, buf_len) \
trace_print_hex_seq(p, buf, buf_len, false)
#undef __print_hex_str
#define __print_hex_str(buf, buf_len) \
trace_print_hex_seq(p, buf, buf_len, true)
#undef __print_array
#define __print_array(array, count, el_size) \
({ \
BUILD_BUG_ON(el_size != 1 && el_size != 2 && \
el_size != 4 && el_size != 8); \
trace_print_array_seq(p, array, count, el_size); \
})
#undef __print_hex_dump
#define __print_hex_dump(prefix_str, prefix_type, \
rowsize, groupsize, buf, len, ascii) \
trace_print_hex_dump_seq(p, prefix_str, prefix_type, \
rowsize, groupsize, buf, len, ascii)
#undef __print_ns_to_secs
#define __print_ns_to_secs(value) \
({ \
u64 ____val = (u64)(value); \
do_div(____val, NSEC_PER_SEC); \
____val; \
})
#undef __print_ns_without_secs
#define __print_ns_without_secs(value) \
({ \
u64 ____val = (u64)(value); \
(u32) do_div(____val, NSEC_PER_SEC); \
})

63
include/trace/stages/stage4_defines.h Normal file
View File

@@ -0,0 +1,63 @@
/* SPDX-License-Identifier: GPL-2.0 */
/* Stage 4 definitions for creating trace events */
#undef __field_ext
#define __field_ext(_type, _item, _filter_type) { \
.type = #_type, .name = #_item, \
.size = sizeof(_type), .align = __alignof__(_type), \
.is_signed = is_signed_type(_type), .filter_type = _filter_type },
#undef __field_struct_ext
#define __field_struct_ext(_type, _item, _filter_type) { \
.type = #_type, .name = #_item, \
.size = sizeof(_type), .align = __alignof__(_type), \
0, .filter_type = _filter_type },
#undef __field
#define __field(type, item) __field_ext(type, item, FILTER_OTHER)
#undef __field_struct
#define __field_struct(type, item) __field_struct_ext(type, item, FILTER_OTHER)
#undef __array
#define __array(_type, _item, _len) { \
.type = #_type"["__stringify(_len)"]", .name = #_item, \
.size = sizeof(_type[_len]), .align = __alignof__(_type), \
.is_signed = is_signed_type(_type), .filter_type = FILTER_OTHER },
#undef __dynamic_array
#define __dynamic_array(_type, _item, _len) { \
.type = "__data_loc " #_type "[]", .name = #_item, \
.size = 4, .align = 4, \
.is_signed = is_signed_type(_type), .filter_type = FILTER_OTHER },
#undef __string
#define __string(item, src) __dynamic_array(char, item, -1)
#undef __string_len
#define __string_len(item, src, len) __dynamic_array(char, item, -1)
#undef __bitmask
#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#undef __rel_dynamic_array
#define __rel_dynamic_array(_type, _item, _len) { \
.type = "__rel_loc " #_type "[]", .name = #_item, \
.size = 4, .align = 4, \
.is_signed = is_signed_type(_type), .filter_type = FILTER_OTHER },
#undef __rel_string
#define __rel_string(item, src) __rel_dynamic_array(char, item, -1)
#undef __rel_string_len
#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, -1)
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, -1)
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)

89
include/trace/stages/stage5_defines.h Normal file
View File

@@ -0,0 +1,89 @@
/* SPDX-License-Identifier: GPL-2.0 */
/* Stage 5 definitions for creating trace events */
/*
* remember the offset of each array from the beginning of the event.
*/
#undef __entry
#define __entry entry
#undef __field
#define __field(type, item)
#undef __field_ext
#define __field_ext(type, item, filter_type)
#undef __field_struct
#define __field_struct(type, item)
#undef __field_struct_ext
#define __field_struct_ext(type, item, filter_type)
#undef __array
#define __array(type, item, len)
#undef __dynamic_array
#define __dynamic_array(type, item, len) \
__item_length = (len) * sizeof(type); \
__data_offsets->item = __data_size + \
offsetof(typeof(*entry), __data); \
__data_offsets->item |= __item_length << 16; \
__data_size += __item_length;
#undef __string
#define __string(item, src) __dynamic_array(char, item, \
strlen((src) ? (const char *)(src) : "(null)") + 1)
#undef __string_len
#define __string_len(item, src, len) __dynamic_array(char, item, (len) + 1)
#undef __rel_dynamic_array
#define __rel_dynamic_array(type, item, len) \
__item_length = (len) * sizeof(type); \
__data_offsets->item = __data_size + \
offsetof(typeof(*entry), __data) - \
offsetof(typeof(*entry), __rel_loc_##item) - \
sizeof(u32); \
__data_offsets->item |= __item_length << 16; \
__data_size += __item_length;
#undef __rel_string
#define __rel_string(item, src) __rel_dynamic_array(char, item, \
strlen((src) ? (const char *)(src) : "(null)") + 1)
#undef __rel_string_len
#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, (len) + 1)
/*
* __bitmask_size_in_bytes_raw is the number of bytes needed to hold
* num_possible_cpus().
*/
#define __bitmask_size_in_bytes_raw(nr_bits) \
(((nr_bits) + 7) / 8)
#define __bitmask_size_in_longs(nr_bits) \
((__bitmask_size_in_bytes_raw(nr_bits) + \
((BITS_PER_LONG / 8) - 1)) / (BITS_PER_LONG / 8))
/*
* __bitmask_size_in_bytes is the number of bytes needed to hold
* num_possible_cpus() padded out to the nearest long. This is what
* is saved in the buffer, just to be consistent.
*/
#define __bitmask_size_in_bytes(nr_bits) \
(__bitmask_size_in_longs(nr_bits) * (BITS_PER_LONG / 8))
#undef __bitmask
#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, \
__bitmask_size_in_longs(nr_bits))
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, \
__bitmask_size_in_longs(nr_bits))
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)

106
include/trace/stages/stage6_defines.h Normal file
View File

@@ -0,0 +1,106 @@
/* SPDX-License-Identifier: GPL-2.0 */
/* Stage 6 definitions for creating trace events */
#undef __entry
#define __entry entry
#undef __field
#define __field(type, item)
#undef __field_struct
#define __field_struct(type, item)
#undef __array
#define __array(type, item, len)
#undef __dynamic_array
#define __dynamic_array(type, item, len) \
__entry->__data_loc_##item = __data_offsets.item;
#undef __string
#define __string(item, src) __dynamic_array(char, item, -1)
#undef __string_len
#define __string_len(item, src, len) __dynamic_array(char, item, -1)
#undef __assign_str
#define __assign_str(dst, src) \
strcpy(__get_str(dst), (src) ? (const char *)(src) : "(null)");
#undef __assign_str_len
#define __assign_str_len(dst, src, len) \
do { \
memcpy(__get_str(dst), (src), (len)); \
__get_str(dst)[len] = '\0'; \
} while(0)
#undef __bitmask
#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
#undef __get_bitmask
#define __get_bitmask(field) (char *)__get_dynamic_array(field)
#undef __assign_bitmask
#define __assign_bitmask(dst, src, nr_bits) \
memcpy(__get_bitmask(dst), (src), __bitmask_size_in_bytes(nr_bits))
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#undef __get_sockaddr
#define __get_sockaddr(field) ((struct sockaddr *)__get_dynamic_array(field))
#undef __assign_sockaddr
#define __assign_sockaddr(dest, src, len) \
memcpy(__get_dynamic_array(dest), src, len)
#undef __rel_dynamic_array
#define __rel_dynamic_array(type, item, len) \
__entry->__rel_loc_##item = __data_offsets.item;
#undef __rel_string
#define __rel_string(item, src) __rel_dynamic_array(char, item, -1)
#undef __rel_string_len
#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, -1)
#undef __assign_rel_str
#define __assign_rel_str(dst, src) \
strcpy(__get_rel_str(dst), (src) ? (const char *)(src) : "(null)");
#undef __assign_rel_str_len
#define __assign_rel_str_len(dst, src, len) \
do { \
memcpy(__get_rel_str(dst), (src), (len)); \
__get_rel_str(dst)[len] = '\0'; \
} while (0)
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, -1)
#undef __get_rel_bitmask
#define __get_rel_bitmask(field) (char *)__get_rel_dynamic_array(field)
#undef __assign_rel_bitmask
#define __assign_rel_bitmask(dst, src, nr_bits) \
memcpy(__get_rel_bitmask(dst), (src), __bitmask_size_in_bytes(nr_bits))
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)
#undef __get_rel_sockaddr
#define __get_rel_sockaddr(field) ((struct sockaddr *)__get_rel_dynamic_array(field))
#undef __assign_rel_sockaddr
#define __assign_rel_sockaddr(dest, src, len) \
memcpy(__get_rel_dynamic_array(dest), src, len)
#undef TP_fast_assign
#define TP_fast_assign(args...) args
#undef __perf_count
#define __perf_count(c) (c)
#undef __perf_task
#define __perf_task(t) (t)

36
include/trace/stages/stage7_defines.h Normal file
View File

@@ -0,0 +1,36 @@
/* SPDX-License-Identifier: GPL-2.0 */
/* Stage 7 definitions for creating trace events */
#undef __entry
#define __entry REC
#undef __print_flags
#undef __print_symbolic
#undef __print_hex
#undef __print_hex_str
#undef __get_dynamic_array
#undef __get_dynamic_array_len
#undef __get_str
#undef __get_bitmask
#undef __get_sockaddr
#undef __get_rel_dynamic_array
#undef __get_rel_dynamic_array_len
#undef __get_rel_str
#undef __get_rel_bitmask
#undef __get_rel_sockaddr
#undef __print_array
#undef __print_hex_dump
/*
* The below is not executed in the kernel. It is only what is
* displayed in the print format for userspace to parse.
*/
#undef __print_ns_to_secs
#define __print_ns_to_secs(val) (val) / 1000000000UL
#undef __print_ns_without_secs
#define __print_ns_without_secs(val) (val) % 1000000000UL
#undef TP_printk
#define TP_printk(fmt, args...) "\"" fmt "\", " __stringify(args)

221
include/trace/trace_custom_events.h Normal file
View File

@@ -0,0 +1,221 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* This is similar to the trace_events.h file, but is to only
* create custom trace events to be attached to existing tracepoints.
* Where as the TRACE_EVENT() macro (from trace_events.h) will create
* both the trace event and the tracepoint it will attach the event to,
* TRACE_CUSTOM_EVENT() is to create only a custom version of an existing
* trace event (created by TRACE_EVENT() or DEFINE_EVENT()), and will
* be placed in the "custom" system.
*/
#include <linux/trace_events.h>
/* All custom events are placed in the custom group */
#undef TRACE_SYSTEM
#define TRACE_SYSTEM custom
#ifndef TRACE_SYSTEM_VAR
#define TRACE_SYSTEM_VAR TRACE_SYSTEM
#endif
/* The init stage creates the system string and enum mappings */
#include "stages/init.h"
#undef TRACE_CUSTOM_EVENT
#define TRACE_CUSTOM_EVENT(name, proto, args, tstruct, assign, print) \
DECLARE_CUSTOM_EVENT_CLASS(name, \
PARAMS(proto), \
PARAMS(args), \
PARAMS(tstruct), \
PARAMS(assign), \
PARAMS(print)); \
DEFINE_CUSTOM_EVENT(name, name, PARAMS(proto), PARAMS(args));
/* Stage 1 creates the structure of the recorded event layout */
#include "stages/stage1_defines.h"
#undef DECLARE_CUSTOM_EVENT_CLASS
#define DECLARE_CUSTOM_EVENT_CLASS(name, proto, args, tstruct, assign, print) \
struct trace_custom_event_raw_##name { \
struct trace_entry ent; \
tstruct \
char __data[]; \
}; \
\
static struct trace_event_class custom_event_class_##name;
#undef DEFINE_CUSTOM_EVENT
#define DEFINE_CUSTOM_EVENT(template, name, proto, args) \
static struct trace_event_call __used \
__attribute__((__aligned__(4))) custom_event_##name
#include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
/* Stage 2 creates the custom class */
#include "stages/stage2_defines.h"
#undef DECLARE_CUSTOM_EVENT_CLASS
#define DECLARE_CUSTOM_EVENT_CLASS(call, proto, args, tstruct, assign, print) \
struct trace_custom_event_data_offsets_##call { \
tstruct; \
};
#undef DEFINE_CUSTOM_EVENT
#define DEFINE_CUSTOM_EVENT(template, name, proto, args)
#include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
/* Stage 3 create the way to print the custom event */
#include "stages/stage3_defines.h"
#undef DECLARE_CUSTOM_EVENT_CLASS
#define DECLARE_CUSTOM_EVENT_CLASS(call, proto, args, tstruct, assign, print) \
static notrace enum print_line_t \
trace_custom_raw_output_##call(struct trace_iterator *iter, int flags, \
struct trace_event *trace_event) \
{ \
struct trace_seq *s = &iter->seq; \
struct trace_seq __maybe_unused *p = &iter->tmp_seq; \
struct trace_custom_event_raw_##call *field; \
int ret; \
\
field = (typeof(field))iter->ent; \
\
ret = trace_raw_output_prep(iter, trace_event); \
if (ret != TRACE_TYPE_HANDLED) \
return ret; \
\
trace_event_printf(iter, print); \
\
return trace_handle_return(s); \
} \
static struct trace_event_functions trace_custom_event_type_funcs_##call = { \
.trace = trace_custom_raw_output_##call, \
};
#include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
/* Stage 4 creates the offset layout for the fields */
#include "stages/stage4_defines.h"
#undef DECLARE_CUSTOM_EVENT_CLASS
#define DECLARE_CUSTOM_EVENT_CLASS(call, proto, args, tstruct, func, print) \
static struct trace_event_fields trace_custom_event_fields_##call[] = { \
tstruct \
{} };
#include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
/* Stage 5 creates the helper function for dynamic fields */
#include "stages/stage5_defines.h"
#undef DECLARE_CUSTOM_EVENT_CLASS
#define DECLARE_CUSTOM_EVENT_CLASS(call, proto, args, tstruct, assign, print) \
static inline notrace int trace_custom_event_get_offsets_##call( \
struct trace_custom_event_data_offsets_##call *__data_offsets, proto) \
{ \
int __data_size = 0; \
int __maybe_unused __item_length; \
struct trace_custom_event_raw_##call __maybe_unused *entry; \
\
tstruct; \
\
return __data_size; \
}
#include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
/* Stage 6 creates the probe function that records the event */
#include "stages/stage6_defines.h"
#undef DECLARE_CUSTOM_EVENT_CLASS
#define DECLARE_CUSTOM_EVENT_CLASS(call, proto, args, tstruct, assign, print) \
\
static notrace void \
trace_custom_event_raw_event_##call(void *__data, proto) \
{ \
struct trace_event_file *trace_file = __data; \
struct trace_custom_event_data_offsets_##call __maybe_unused __data_offsets; \
struct trace_event_buffer fbuffer; \
struct trace_custom_event_raw_##call *entry; \
int __data_size; \
\
if (trace_trigger_soft_disabled(trace_file)) \
return; \
\
__data_size = trace_custom_event_get_offsets_##call(&__data_offsets, args); \
\
entry = trace_event_buffer_reserve(&fbuffer, trace_file, \
sizeof(*entry) + __data_size); \
\
if (!entry) \
return; \
\
tstruct \
\
{ assign; } \
\
trace_event_buffer_commit(&fbuffer); \
}
/*
* The ftrace_test_custom_probe is compiled out, it is only here as a build time check
* to make sure that if the tracepoint handling changes, the ftrace probe will
* fail to compile unless it too is updated.
*/
#undef DEFINE_CUSTOM_EVENT
#define DEFINE_CUSTOM_EVENT(template, call, proto, args) \
static inline void ftrace_test_custom_probe_##call(void) \
{ \
check_trace_callback_type_##call(trace_custom_event_raw_event_##template); \
}
#include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
/* Stage 7 creates the actual class and event structure for the custom event */
#include "stages/stage7_defines.h"
#undef DECLARE_CUSTOM_EVENT_CLASS
#define DECLARE_CUSTOM_EVENT_CLASS(call, proto, args, tstruct, assign, print) \
static char custom_print_fmt_##call[] = print; \
static struct trace_event_class __used __refdata custom_event_class_##call = { \
.system = TRACE_SYSTEM_STRING, \
.fields_array = trace_custom_event_fields_##call, \
.fields = LIST_HEAD_INIT(custom_event_class_##call.fields),\
.raw_init = trace_event_raw_init, \
.probe = trace_custom_event_raw_event_##call, \
.reg = trace_event_reg, \
};
#undef DEFINE_CUSTOM_EVENT
#define DEFINE_CUSTOM_EVENT(template, call, proto, args) \
\
static struct trace_event_call __used custom_event_##call = { \
.name = #call, \
.class = &custom_event_class_##template, \
.event.funcs = &trace_custom_event_type_funcs_##template, \
.print_fmt = custom_print_fmt_##template, \
.flags = TRACE_EVENT_FL_CUSTOM, \
}; \
static inline int trace_custom_event_##call##_update(struct tracepoint *tp) \
{ \
if (tp->name && strcmp(tp->name, #call) == 0) { \
custom_event_##call.tp = tp; \
custom_event_##call.flags = TRACE_EVENT_FL_TRACEPOINT; \
return 1; \
} \
return 0; \
} \
static struct trace_event_call __used \
__section("_ftrace_events") *__custom_event_##call = &custom_event_##call
#include TRACE_INCLUDE(TRACE_INCLUDE_FILE)

550
include/trace/trace_events.h
View File

@@ -24,42 +24,7 @@
#define TRACE_SYSTEM_VAR TRACE_SYSTEM
#endif
#define __app__(x, y) str__##x##y
#define __app(x, y) __app__(x, y)
#define TRACE_SYSTEM_STRING __app(TRACE_SYSTEM_VAR,__trace_system_name)
#define TRACE_MAKE_SYSTEM_STR() \
static const char TRACE_SYSTEM_STRING[] = \
__stringify(TRACE_SYSTEM)
TRACE_MAKE_SYSTEM_STR();
#undef TRACE_DEFINE_ENUM
#define TRACE_DEFINE_ENUM(a) \
static struct trace_eval_map __used __initdata \
__##TRACE_SYSTEM##_##a = \
{ \
.system = TRACE_SYSTEM_STRING, \
.eval_string = #a, \
.eval_value = a \
}; \
static struct trace_eval_map __used \
__section("_ftrace_eval_map") \
*TRACE_SYSTEM##_##a = &__##TRACE_SYSTEM##_##a
#undef TRACE_DEFINE_SIZEOF
#define TRACE_DEFINE_SIZEOF(a) \
static struct trace_eval_map __used __initdata \
__##TRACE_SYSTEM##_##a = \
{ \
.system = TRACE_SYSTEM_STRING, \
.eval_string = "sizeof(" #a ")", \
.eval_value = sizeof(a) \
}; \
static struct trace_eval_map __used \
__section("_ftrace_eval_map") \
*TRACE_SYSTEM##_##a = &__##TRACE_SYSTEM##_##a
#include "stages/init.h"
/*
* DECLARE_EVENT_CLASS can be used to add a generic function
@@ -80,54 +45,7 @@ TRACE_MAKE_SYSTEM_STR();
PARAMS(print)); \
DEFINE_EVENT(name, name, PARAMS(proto), PARAMS(args));
#undef __field
#define __field(type, item) type item;
#undef __field_ext
#define __field_ext(type, item, filter_type) type item;
#undef __field_struct
#define __field_struct(type, item) type item;
#undef __field_struct_ext
#define __field_struct_ext(type, item, filter_type) type item;
#undef __array
#define __array(type, item, len) type item[len];
#undef __dynamic_array
#define __dynamic_array(type, item, len) u32 __data_loc_##item;
#undef __string
#define __string(item, src) __dynamic_array(char, item, -1)
#undef __string_len
#define __string_len(item, src, len) __dynamic_array(char, item, -1)
#undef __bitmask
#define __bitmask(item, nr_bits) __dynamic_array(char, item, -1)
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#undef __rel_dynamic_array
#define __rel_dynamic_array(type, item, len) u32 __rel_loc_##item;
#undef __rel_string
#define __rel_string(item, src) __rel_dynamic_array(char, item, -1)
#undef __rel_string_len
#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, -1)
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(char, item, -1)
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)
#undef TP_STRUCT__entry
#define TP_STRUCT__entry(args...) args
#include "stages/stage1_defines.h"
#undef DECLARE_EVENT_CLASS
#define DECLARE_EVENT_CLASS(name, proto, args, tstruct, assign, print) \
@@ -191,56 +109,7 @@ TRACE_MAKE_SYSTEM_STR();
* The size of an array is also encoded, in the higher 16 bits of <item>.
*/
#undef TRACE_DEFINE_ENUM
#define TRACE_DEFINE_ENUM(a)
#undef TRACE_DEFINE_SIZEOF
#define TRACE_DEFINE_SIZEOF(a)
#undef __field
#define __field(type, item)
#undef __field_ext
#define __field_ext(type, item, filter_type)
#undef __field_struct
#define __field_struct(type, item)
#undef __field_struct_ext
#define __field_struct_ext(type, item, filter_type)
#undef __array
#define __array(type, item, len)
#undef __dynamic_array
#define __dynamic_array(type, item, len) u32 item;
#undef __string
#define __string(item, src) __dynamic_array(char, item, -1)
#undef __string_len
#define __string_len(item, src, len) __dynamic_array(char, item, -1)
#undef __bitmask
#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#undef __rel_dynamic_array
#define __rel_dynamic_array(type, item, len) u32 item;
#undef __rel_string
#define __rel_string(item, src) __rel_dynamic_array(char, item, -1)
#undef __rel_string_len
#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, -1)
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, -1)
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)
#include "stages/stage2_defines.h"
#undef DECLARE_EVENT_CLASS
#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print) \
@@ -312,137 +181,7 @@ TRACE_MAKE_SYSTEM_STR();
* in binary.
*/
#undef __entry
#define __entry field
#undef TP_printk
#define TP_printk(fmt, args...) fmt "\n", args
#undef __get_dynamic_array
#define __get_dynamic_array(field) \
((void *)__entry + (__entry->__data_loc_##field & 0xffff))
#undef __get_dynamic_array_len
#define __get_dynamic_array_len(field) \
((__entry->__data_loc_##field >> 16) & 0xffff)
#undef __get_str
#define __get_str(field) ((char *)__get_dynamic_array(field))
#undef __get_rel_dynamic_array
#define __get_rel_dynamic_array(field) \
((void *)__entry + \
offsetof(typeof(*__entry), __rel_loc_##field) + \
sizeof(__entry->__rel_loc_##field) + \
(__entry->__rel_loc_##field & 0xffff))
#undef __get_rel_dynamic_array_len
#define __get_rel_dynamic_array_len(field) \
((__entry->__rel_loc_##field >> 16) & 0xffff)
#undef __get_rel_str
#define __get_rel_str(field) ((char *)__get_rel_dynamic_array(field))
#undef __get_bitmask
#define __get_bitmask(field) \
({ \
void *__bitmask = __get_dynamic_array(field); \
unsigned int __bitmask_size; \
__bitmask_size = __get_dynamic_array_len(field); \
trace_print_bitmask_seq(p, __bitmask, __bitmask_size); \
})
#undef __get_rel_bitmask
#define __get_rel_bitmask(field) \
({ \
void *__bitmask = __get_rel_dynamic_array(field); \
unsigned int __bitmask_size; \
__bitmask_size = __get_rel_dynamic_array_len(field); \
trace_print_bitmask_seq(p, __bitmask, __bitmask_size); \
})
#undef __get_sockaddr
#define __get_sockaddr(field) ((struct sockaddr *)__get_dynamic_array(field))
#undef __get_rel_sockaddr
#define __get_rel_sockaddr(field) ((struct sockaddr *)__get_rel_dynamic_array(field))
#undef __print_flags
#define __print_flags(flag, delim, flag_array...) \
({ \
static const struct trace_print_flags __flags[] = \
{ flag_array, { -1, NULL }}; \
trace_print_flags_seq(p, delim, flag, __flags); \
})
#undef __print_symbolic
#define __print_symbolic(value, symbol_array...) \
({ \
static const struct trace_print_flags symbols[] = \
{ symbol_array, { -1, NULL }}; \
trace_print_symbols_seq(p, value, symbols); \
})
#undef __print_flags_u64
#undef __print_symbolic_u64
#if BITS_PER_LONG == 32
#define __print_flags_u64(flag, delim, flag_array...) \
({ \
static const struct trace_print_flags_u64 __flags[] = \
{ flag_array, { -1, NULL } }; \
trace_print_flags_seq_u64(p, delim, flag, __flags); \
})
#define __print_symbolic_u64(value, symbol_array...) \
({ \
static const struct trace_print_flags_u64 symbols[] = \
{ symbol_array, { -1, NULL } }; \
trace_print_symbols_seq_u64(p, value, symbols); \
})
#else
#define __print_flags_u64(flag, delim, flag_array...) \
__print_flags(flag, delim, flag_array)
#define __print_symbolic_u64(value, symbol_array...) \
__print_symbolic(value, symbol_array)
#endif
#undef __print_hex
#define __print_hex(buf, buf_len) \
trace_print_hex_seq(p, buf, buf_len, false)
#undef __print_hex_str
#define __print_hex_str(buf, buf_len) \
trace_print_hex_seq(p, buf, buf_len, true)
#undef __print_array
#define __print_array(array, count, el_size) \
({ \
BUILD_BUG_ON(el_size != 1 && el_size != 2 && \
el_size != 4 && el_size != 8); \
trace_print_array_seq(p, array, count, el_size); \
})
#undef __print_hex_dump
#define __print_hex_dump(prefix_str, prefix_type, \
rowsize, groupsize, buf, len, ascii) \
trace_print_hex_dump_seq(p, prefix_str, prefix_type, \
rowsize, groupsize, buf, len, ascii)
#undef __print_ns_to_secs
#define __print_ns_to_secs(value) \
({ \
u64 ____val = (u64)(value); \
do_div(____val, NSEC_PER_SEC); \
____val; \
})
#undef __print_ns_without_secs
#define __print_ns_without_secs(value) \
({ \
u64 ____val = (u64)(value); \
(u32) do_div(____val, NSEC_PER_SEC); \
})
#include "stages/stage3_defines.h"
#undef DECLARE_EVENT_CLASS
#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print) \
@@ -497,65 +236,7 @@ static struct trace_event_functions trace_event_type_funcs_##call = { \
#include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
#undef __field_ext
#define __field_ext(_type, _item, _filter_type) { \
.type = #_type, .name = #_item, \
.size = sizeof(_type), .align = __alignof__(_type), \
.is_signed = is_signed_type(_type), .filter_type = _filter_type },
#undef __field_struct_ext
#define __field_struct_ext(_type, _item, _filter_type) { \
.type = #_type, .name = #_item, \
.size = sizeof(_type), .align = __alignof__(_type), \
0, .filter_type = _filter_type },
#undef __field
#define __field(type, item) __field_ext(type, item, FILTER_OTHER)
#undef __field_struct
#define __field_struct(type, item) __field_struct_ext(type, item, FILTER_OTHER)
#undef __array
#define __array(_type, _item, _len) { \
.type = #_type"["__stringify(_len)"]", .name = #_item, \
.size = sizeof(_type[_len]), .align = __alignof__(_type), \
.is_signed = is_signed_type(_type), .filter_type = FILTER_OTHER },
#undef __dynamic_array
#define __dynamic_array(_type, _item, _len) { \
.type = "__data_loc " #_type "[]", .name = #_item, \
.size = 4, .align = 4, \
.is_signed = is_signed_type(_type), .filter_type = FILTER_OTHER },
#undef __string
#define __string(item, src) __dynamic_array(char, item, -1)
#undef __string_len
#define __string_len(item, src, len) __dynamic_array(char, item, -1)
#undef __bitmask
#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#undef __rel_dynamic_array
#define __rel_dynamic_array(_type, _item, _len) { \
.type = "__rel_loc " #_type "[]", .name = #_item, \
.size = 4, .align = 4, \
.is_signed = is_signed_type(_type), .filter_type = FILTER_OTHER },
#undef __rel_string
#define __rel_string(item, src) __rel_dynamic_array(char, item, -1)
#undef __rel_string_len
#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, -1)
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, -1)
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)
#include "stages/stage4_defines.h"
#undef DECLARE_EVENT_CLASS
#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, func, print) \
@@ -568,91 +249,7 @@ static struct trace_event_fields trace_event_fields_##call[] = { \
#include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
/*
* remember the offset of each array from the beginning of the event.
*/
#undef __entry
#define __entry entry
#undef __field
#define __field(type, item)
#undef __field_ext
#define __field_ext(type, item, filter_type)
#undef __field_struct
#define __field_struct(type, item)
#undef __field_struct_ext
#define __field_struct_ext(type, item, filter_type)
#undef __array
#define __array(type, item, len)
#undef __dynamic_array
#define __dynamic_array(type, item, len) \
__item_length = (len) * sizeof(type); \
__data_offsets->item = __data_size + \
offsetof(typeof(*entry), __data); \
__data_offsets->item |= __item_length << 16; \
__data_size += __item_length;
#undef __string
#define __string(item, src) __dynamic_array(char, item, \
strlen((src) ? (const char *)(src) : "(null)") + 1)
#undef __string_len
#define __string_len(item, src, len) __dynamic_array(char, item, (len) + 1)
#undef __rel_dynamic_array
#define __rel_dynamic_array(type, item, len) \
__item_length = (len) * sizeof(type); \
__data_offsets->item = __data_size + \
offsetof(typeof(*entry), __data) - \
offsetof(typeof(*entry), __rel_loc_##item) - \
sizeof(u32); \
__data_offsets->item |= __item_length << 16; \
__data_size += __item_length;
#undef __rel_string
#define __rel_string(item, src) __rel_dynamic_array(char, item, \
strlen((src) ? (const char *)(src) : "(null)") + 1)
#undef __rel_string_len
#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, (len) + 1)
/*
* __bitmask_size_in_bytes_raw is the number of bytes needed to hold
* num_possible_cpus().
*/
#define __bitmask_size_in_bytes_raw(nr_bits) \
(((nr_bits) + 7) / 8)
#define __bitmask_size_in_longs(nr_bits) \
((__bitmask_size_in_bytes_raw(nr_bits) + \
((BITS_PER_LONG / 8) - 1)) / (BITS_PER_LONG / 8))
/*
* __bitmask_size_in_bytes is the number of bytes needed to hold
* num_possible_cpus() padded out to the nearest long. This is what
* is saved in the buffer, just to be consistent.
*/
#define __bitmask_size_in_bytes(nr_bits) \
(__bitmask_size_in_longs(nr_bits) * (BITS_PER_LONG / 8))
#undef __bitmask
#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, \
__bitmask_size_in_longs(nr_bits))
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, \
__bitmask_size_in_longs(nr_bits))
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)
#include "stages/stage5_defines.h"
#undef DECLARE_EVENT_CLASS
#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print) \
@@ -775,107 +372,7 @@ static inline notrace int trace_event_get_offsets_##call( \
#define _TRACE_PERF_INIT(call)
#endif /* CONFIG_PERF_EVENTS */
#undef __entry
#define __entry entry
#undef __field
#define __field(type, item)
#undef __field_struct
#define __field_struct(type, item)
#undef __array
#define __array(type, item, len)
#undef __dynamic_array
#define __dynamic_array(type, item, len) \
__entry->__data_loc_##item = __data_offsets.item;
#undef __string
#define __string(item, src) __dynamic_array(char, item, -1)
#undef __string_len
#define __string_len(item, src, len) __dynamic_array(char, item, -1)
#undef __assign_str
#define __assign_str(dst, src) \
strcpy(__get_str(dst), (src) ? (const char *)(src) : "(null)");
#undef __assign_str_len
#define __assign_str_len(dst, src, len) \
do { \
memcpy(__get_str(dst), (src), (len)); \
__get_str(dst)[len] = '\0'; \
} while(0)
#undef __bitmask
#define __bitmask(item, nr_bits) __dynamic_array(unsigned long, item, -1)
#undef __get_bitmask
#define __get_bitmask(field) (char *)__get_dynamic_array(field)
#undef __assign_bitmask
#define __assign_bitmask(dst, src, nr_bits) \
memcpy(__get_bitmask(dst), (src), __bitmask_size_in_bytes(nr_bits))
#undef __sockaddr
#define __sockaddr(field, len) __dynamic_array(u8, field, len)
#undef __get_sockaddr
#define __get_sockaddr(field) ((struct sockaddr *)__get_dynamic_array(field))
#define __assign_sockaddr(dest, src, len) \
memcpy(__get_dynamic_array(dest), src, len)
#undef __rel_dynamic_array
#define __rel_dynamic_array(type, item, len) \
__entry->__rel_loc_##item = __data_offsets.item;
#undef __rel_string
#define __rel_string(item, src) __rel_dynamic_array(char, item, -1)
#undef __rel_string_len
#define __rel_string_len(item, src, len) __rel_dynamic_array(char, item, -1)
#undef __assign_rel_str
#define __assign_rel_str(dst, src) \
strcpy(__get_rel_str(dst), (src) ? (const char *)(src) : "(null)");
#undef __assign_rel_str_len
#define __assign_rel_str_len(dst, src, len) \
do { \
memcpy(__get_rel_str(dst), (src), (len)); \
__get_rel_str(dst)[len] = '\0'; \
} while (0)
#undef __rel_bitmask
#define __rel_bitmask(item, nr_bits) __rel_dynamic_array(unsigned long, item, -1)
#undef __get_rel_bitmask
#define __get_rel_bitmask(field) (char *)__get_rel_dynamic_array(field)
#undef __assign_rel_bitmask
#define __assign_rel_bitmask(dst, src, nr_bits) \
memcpy(__get_rel_bitmask(dst), (src), __bitmask_size_in_bytes(nr_bits))
#undef __rel_sockaddr
#define __rel_sockaddr(field, len) __rel_dynamic_array(u8, field, len)
#undef __get_rel_sockaddr
#define __get_rel_sockaddr(field) ((struct sockaddr *)__get_rel_dynamic_array(field))
#define __assign_rel_sockaddr(dest, src, len) \
memcpy(__get_rel_dynamic_array(dest), src, len)
#undef TP_fast_assign
#define TP_fast_assign(args...) args
#undef __perf_count
#define __perf_count(c) (c)
#undef __perf_task
#define __perf_task(t) (t)
#include "stages/stage6_defines.h"
#undef DECLARE_EVENT_CLASS
#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print) \
@@ -921,38 +418,7 @@ static inline void ftrace_test_probe_##call(void) \
#include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
#undef __entry
#define __entry REC
#undef __print_flags
#undef __print_symbolic
#undef __print_hex
#undef __print_hex_str
#undef __get_dynamic_array
#undef __get_dynamic_array_len
#undef __get_str
#undef __get_bitmask
#undef __get_sockaddr
#undef __get_rel_dynamic_array
#undef __get_rel_dynamic_array_len
#undef __get_rel_str
#undef __get_rel_bitmask
#undef __get_rel_sockaddr
#undef __print_array
#undef __print_hex_dump
/*
* The below is not executed in the kernel. It is only what is
* displayed in the print format for userspace to parse.
*/
#undef __print_ns_to_secs
#define __print_ns_to_secs(val) (val) / 1000000000UL
#undef __print_ns_without_secs
#define __print_ns_without_secs(val) (val) % 1000000000UL
#undef TP_printk
#define TP_printk(fmt, args...) "\"" fmt "\", " __stringify(args)
#include "stages/stage7_defines.h"
#undef DECLARE_EVENT_CLASS
#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print) \

116
include/uapi/linux/user_events.h Normal file
View File

@@ -0,0 +1,116 @@
/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
/*
* Copyright (c) 2021, Microsoft Corporation.
*
* Authors:
* Beau Belgrave <beaub@linux.microsoft.com>
*/
#ifndef _UAPI_LINUX_USER_EVENTS_H
#define _UAPI_LINUX_USER_EVENTS_H
#include <linux/types.h>
#include <linux/ioctl.h>
#ifdef __KERNEL__
#include <linux/uio.h>
#else
#include <sys/uio.h>
#endif
#define USER_EVENTS_SYSTEM "user_events"
#define USER_EVENTS_PREFIX "u:"
/* Bits 0-6 are for known probe types, Bit 7 is for unknown probes */
#define EVENT_BIT_FTRACE 0
#define EVENT_BIT_PERF 1
#define EVENT_BIT_OTHER 7
#define EVENT_STATUS_FTRACE (1 << EVENT_BIT_FTRACE)
#define EVENT_STATUS_PERF (1 << EVENT_BIT_PERF)
#define EVENT_STATUS_OTHER (1 << EVENT_BIT_OTHER)
/* Create dynamic location entry within a 32-bit value */
#define DYN_LOC(offset, size) ((size) << 16 | (offset))
/* Use raw iterator for attached BPF program(s), no affect on ftrace/perf */
#define FLAG_BPF_ITER (1 << 0)
/*
* Describes an event registration and stores the results of the registration.
* This structure is passed to the DIAG_IOCSREG ioctl, callers at a minimum
* must set the size and name_args before invocation.
*/
struct user_reg {
/* Input: Size of the user_reg structure being used */
__u32 size;
/* Input: Pointer to string with event name, description and flags */
__u64 name_args;
/* Output: Byte index of the event within the status page */
__u32 status_index;
/* Output: Index of the event to use when writing data */
__u32 write_index;
};
#define DIAG_IOC_MAGIC '*'
/* Requests to register a user_event */
#define DIAG_IOCSREG _IOWR(DIAG_IOC_MAGIC, 0, struct user_reg*)
/* Requests to delete a user_event */
#define DIAG_IOCSDEL _IOW(DIAG_IOC_MAGIC, 1, char*)
/* Data type that was passed to the BPF program */
enum {
/* Data resides in kernel space */
USER_BPF_DATA_KERNEL,
/* Data resides in user space */
USER_BPF_DATA_USER,
/* Data is a pointer to a user_bpf_iter structure */
USER_BPF_DATA_ITER,
};
/*
* Describes an iovec iterator that BPF programs can use to access data for
* a given user_event write() / writev() call.
*/
struct user_bpf_iter {
/* Offset of the data within the first iovec */
__u32 iov_offset;
/* Number of iovec structures */
__u32 nr_segs;
/* Pointer to iovec structures */
const struct iovec *iov;
};
/* Context that BPF programs receive when attached to a user_event */
struct user_bpf_context {
/* Data type being passed (see union below) */
__u32 data_type;
/* Length of the data */
__u32 data_len;
/* Pointer to data, varies by data type */
union {
/* Kernel data (data_type == USER_BPF_DATA_KERNEL) */
void *kdata;
/* User data (data_type == USER_BPF_DATA_USER) */
void *udata;
/* Direct iovec (data_type == USER_BPF_DATA_ITER) */
struct user_bpf_iter *iter;
};
};
#endif /* _UAPI_LINUX_USER_EVENTS_H */

85
kernel/printk/printk.c
View File

@@ -95,6 +95,12 @@ EXPORT_SYMBOL_GPL(console_drivers);
*/
int __read_mostly suppress_printk;
/*
* During panic, heavy printk by other CPUs can delay the
* panic and risk deadlock on console resources.
*/
static int __read_mostly suppress_panic_printk;
#ifdef CONFIG_LOCKDEP
static struct lockdep_map console_lock_dep_map = {
.name = "console_lock"
@@ -148,8 +154,10 @@ static int __control_devkmsg(char *str)
static int __init control_devkmsg(char *str)
{
if (__control_devkmsg(str) < 0)
if (__control_devkmsg(str) < 0) {
pr_warn("printk.devkmsg: bad option string '%s'\n", str);
return 1;
}
/*
* Set sysctl string accordingly:
@@ -168,7 +176,7 @@ static int __init control_devkmsg(char *str)
*/
devkmsg_log |= DEVKMSG_LOG_MASK_LOCK;
return 0;
return 1;
}
__setup("printk.devkmsg=", control_devkmsg);
@@ -259,6 +267,11 @@ static void __up_console_sem(unsigned long ip)
}
#define up_console_sem() __up_console_sem(_RET_IP_)
static bool panic_in_progress(void)
{
return unlikely(atomic_read(&panic_cpu) != PANIC_CPU_INVALID);
}
/*
* This is used for debugging the mess that is the VT code by
* keeping track if we have the console semaphore held. It's
@@ -1845,6 +1858,16 @@ static int console_trylock_spinning(void)
if (console_trylock())
return 1;
/*
* It's unsafe to spin once a panic has begun. If we are the
* panic CPU, we may have already halted the owner of the
* console_sem. If we are not the panic CPU, then we should
* avoid taking console_sem, so the panic CPU has a better
* chance of cleanly acquiring it later.
*/
if (panic_in_progress())
return 0;
printk_safe_enter_irqsave(flags);
raw_spin_lock(&console_owner_lock);
@@ -2220,6 +2243,10 @@ asmlinkage int vprintk_emit(int facility, int level,
if (unlikely(suppress_printk))
return 0;
if (unlikely(suppress_panic_printk) &&
atomic_read(&panic_cpu) != raw_smp_processor_id())
return 0;
if (level == LOGLEVEL_SCHED) {
level = LOGLEVEL_DEFAULT;
in_sched = true;
@@ -2326,6 +2353,20 @@ asmlinkage __visible void early_printk(const char *fmt, ...)
}
#endif
static void set_user_specified(struct console_cmdline *c, bool user_specified)
{
if (!user_specified)
return;
/*
* @c console was defined by the user on the command line.
* Do not clear when added twice also by SPCR or the device tree.
*/
c->user_specified = true;
/* At least one console defined by the user on the command line. */
console_set_on_cmdline = 1;
}
static int __add_preferred_console(char *name, int idx, char *options,
char *brl_options, bool user_specified)
{
@@ -2342,8 +2383,7 @@ static int __add_preferred_console(char *name, int idx, char *options,
if (strcmp(c->name, name) == 0 && c->index == idx) {
if (!brl_options)
preferred_console = i;
if (user_specified)
c->user_specified = true;
set_user_specified(c, user_specified);
return 0;
}
}
@@ -2353,7 +2393,7 @@ static int __add_preferred_console(char *name, int idx, char *options,
preferred_console = i;
strlcpy(c->name, name, sizeof(c->name));
c->options = options;
c->user_specified = user_specified;
set_user_specified(c, user_specified);
braille_set_options(c, brl_options);
c->index = idx;
@@ -2419,7 +2459,6 @@ static int __init console_setup(char *str)
*s = 0;
__add_preferred_console(buf, idx, options, brl_options, true);
console_set_on_cmdline = 1;
return 1;
}
__setup("console=", console_setup);
@@ -2581,6 +2620,25 @@ static int have_callable_console(void)
return 0;
}
/*
* Return true when this CPU should unlock console_sem without pushing all
* messages to the console. This reduces the chance that the console is
* locked when the panic CPU tries to use it.
*/
static bool abandon_console_lock_in_panic(void)
{
if (!panic_in_progress())
return false;
/*
* We can use raw_smp_processor_id() here because it is impossible for
* the task to be migrated to the panic_cpu, or away from it. If
* panic_cpu has already been set, and we're not currently executing on
* that CPU, then we never will be.
*/
return atomic_read(&panic_cpu) != raw_smp_processor_id();
}
/*
* Can we actually use the console at this time on this cpu?
*
@@ -2611,6 +2669,7 @@ void console_unlock(void)
{
static char ext_text[CONSOLE_EXT_LOG_MAX];
static char text[CONSOLE_LOG_MAX];
static int panic_console_dropped;
unsigned long flags;
bool do_cond_resched, retry;
struct printk_info info;
@@ -2665,6 +2724,10 @@ skip:
if (console_seq != r.info->seq) {
console_dropped += r.info->seq - console_seq;
console_seq = r.info->seq;
if (panic_in_progress() && panic_console_dropped++ > 10) {
suppress_panic_printk = 1;
pr_warn_once("Too many dropped messages. Suppress messages on non-panic CPUs to prevent livelock.\n");
}
}
if (suppress_message_printing(r.info->level)) {
@@ -2724,6 +2787,10 @@ skip:
if (handover)
return;
/* Allow panic_cpu to take over the consoles safely */
if (abandon_console_lock_in_panic())
break;
if (do_cond_resched)
cond_resched();
}
@@ -2741,7 +2808,7 @@ skip:
* flush, no worries.
*/
retry = prb_read_valid(prb, next_seq, NULL);
if (retry && console_trylock())
if (retry && !abandon_console_lock_in_panic() && console_trylock())
goto again;
}
EXPORT_SYMBOL(console_unlock);
@@ -3236,7 +3303,7 @@ static DEFINE_PER_CPU(int, printk_pending);
static void wake_up_klogd_work_func(struct irq_work *irq_work)
{
int pending = __this_cpu_xchg(printk_pending, 0);
int pending = this_cpu_xchg(printk_pending, 0);
if (pending & PRINTK_PENDING_OUTPUT) {
/* If trylock fails, someone else is doing the printing */
@@ -3270,7 +3337,7 @@ void defer_console_output(void)
return;
preempt_disable();
__this_cpu_or(printk_pending, PRINTK_PENDING_OUTPUT);
this_cpu_or(printk_pending, PRINTK_PENDING_OUTPUT);
irq_work_queue(this_cpu_ptr(&wake_up_klogd_work));
preempt_enable();
}

52
kernel/printk/printk_ringbuffer.c
View File

@@ -474,8 +474,10 @@ static enum desc_state desc_read(struct prb_desc_ring *desc_ring,
* state has been re-checked. A memcpy() for all of @desc
* cannot be used because of the atomic_t @state_var field.
*/
memcpy(&desc_out->text_blk_lpos, &desc->text_blk_lpos,
sizeof(desc_out->text_blk_lpos)); /* LMM(desc_read:C) */
if (desc_out) {
memcpy(&desc_out->text_blk_lpos, &desc->text_blk_lpos,
sizeof(desc_out->text_blk_lpos)); /* LMM(desc_read:C) */
}
if (seq_out)
*seq_out = info->seq; /* also part of desc_read:C */
if (caller_id_out)
@@ -528,7 +530,8 @@ static enum desc_state desc_read(struct prb_desc_ring *desc_ring,
state_val = atomic_long_read(state_var); /* LMM(desc_read:E) */
d_state = get_desc_state(id, state_val);
out:
atomic_long_set(&desc_out->state_var, state_val);
if (desc_out)
atomic_long_set(&desc_out->state_var, state_val);
return d_state;
}
@@ -1449,6 +1452,9 @@ static void desc_make_final(struct prb_desc_ring *desc_ring, unsigned long id)
atomic_long_cmpxchg_relaxed(&d->state_var, prev_state_val,
DESC_SV(id, desc_finalized)); /* LMM(desc_make_final:A) */
/* Best effort to remember the last finalized @id. */
atomic_long_set(&desc_ring->last_finalized_id, id);
}
/**
@@ -1657,7 +1663,12 @@ void prb_commit(struct prb_reserved_entry *e)
*/
void prb_final_commit(struct prb_reserved_entry *e)
{
struct prb_desc_ring *desc_ring = &e->rb->desc_ring;
_prb_commit(e, desc_finalized);
/* Best effort to remember the last finalized @id. */
atomic_long_set(&desc_ring->last_finalized_id, e->id);
}
/*
@@ -2005,9 +2016,39 @@ u64 prb_first_valid_seq(struct printk_ringbuffer *rb)
*/
u64 prb_next_seq(struct printk_ringbuffer *rb)
{
u64 seq = 0;
struct prb_desc_ring *desc_ring = &rb->desc_ring;
enum desc_state d_state;
unsigned long id;
u64 seq;
/* Search forward from the oldest descriptor. */
/* Check if the cached @id still points to a valid @seq. */
id = atomic_long_read(&desc_ring->last_finalized_id);
d_state = desc_read(desc_ring, id, NULL, &seq, NULL);
if (d_state == desc_finalized || d_state == desc_reusable) {
/*
* Begin searching after the last finalized record.
*
* On 0, the search must begin at 0 because of hack#2
* of the bootstrapping phase it is not known if a
* record at index 0 exists.
*/
if (seq != 0)
seq++;
} else {
/*
* The information about the last finalized sequence number
* has gone. It should happen only when there is a flood of
* new messages and the ringbuffer is rapidly recycled.
* Give up and start from the beginning.
*/
seq = 0;
}
/*
* The information about the last finalized @seq might be inaccurate.
* Search forward to find the current one.
*/
while (_prb_read_valid(rb, &seq, NULL, NULL))
seq++;
@@ -2044,6 +2085,7 @@ void prb_init(struct printk_ringbuffer *rb,
rb->desc_ring.infos = infos;
atomic_long_set(&rb->desc_ring.head_id, DESC0_ID(descbits));
atomic_long_set(&rb->desc_ring.tail_id, DESC0_ID(descbits));
atomic_long_set(&rb->desc_ring.last_finalized_id, DESC0_ID(descbits));
rb->text_data_ring.size_bits = textbits;
rb->text_data_ring.data = text_buf;

2
kernel/printk/printk_ringbuffer.h
View File

@@ -75,6 +75,7 @@ struct prb_desc_ring {
struct printk_info *infos;
atomic_long_t head_id;
atomic_long_t tail_id;
atomic_long_t last_finalized_id;
};
/*
@@ -258,6 +259,7 @@ static struct printk_ringbuffer name = { \
.infos = &_##name##_infos[0], \
.head_id = ATOMIC_INIT(DESC0_ID(descbits)), \
.tail_id = ATOMIC_INIT(DESC0_ID(descbits)), \
.last_finalized_id = ATOMIC_INIT(DESC0_ID(descbits)), \
}, \
.text_data_ring = { \
.size_bits = (avgtextbits) + (descbits), \

14
kernel/trace/Kconfig
View File

@@ -737,6 +737,20 @@ config SYNTH_EVENTS
If in doubt, say N.
config USER_EVENTS
bool "User trace events"
select TRACING
select DYNAMIC_EVENTS
help
User trace events are user-defined trace events that
can be used like an existing kernel trace event. User trace
events are generated by writing to a tracefs file. User
processes can determine if their tracing events should be
generated by memory mapping a tracefs file and checking for
an associated byte being non-zero.
If in doubt, say N.
config HIST_TRIGGERS
bool "Histogram triggers"
depends on ARCH_HAVE_NMI_SAFE_CMPXCHG

1
kernel/trace/Makefile
View File

@@ -82,6 +82,7 @@ obj-$(CONFIG_PROBE_EVENTS) += trace_eprobe.o
obj-$(CONFIG_TRACE_EVENT_INJECT) += trace_events_inject.o
obj-$(CONFIG_SYNTH_EVENTS) += trace_events_synth.o
obj-$(CONFIG_HIST_TRIGGERS) += trace_events_hist.o
obj-$(CONFIG_USER_EVENTS) += trace_events_user.o
obj-$(CONFIG_BPF_EVENTS) += bpf_trace.o
obj-$(CONFIG_KPROBE_EVENTS) += trace_kprobe.o
obj-$(CONFIG_TRACEPOINTS) += error_report-traces.o

2
kernel/trace/ftrace.c
View File

@@ -7096,6 +7096,8 @@ void __init ftrace_free_init_mem(void)
void *start = (void *)(&__init_begin);
void *end = (void *)(&__init_end);
ftrace_boot_snapshot();
ftrace_free_mem(NULL, start, end);
}

73
kernel/trace/trace.c
View File

@@ -186,6 +186,7 @@ static char bootup_tracer_buf[MAX_TRACER_SIZE] __initdata;
static char *default_bootup_tracer;
static bool allocate_snapshot;
static bool snapshot_at_boot;
static int __init set_cmdline_ftrace(char *str)
{
@@ -231,6 +232,15 @@ static int __init boot_alloc_snapshot(char *str)
__setup("alloc_snapshot", boot_alloc_snapshot);
static int __init boot_snapshot(char *str)
{
snapshot_at_boot = true;
boot_alloc_snapshot(str);
return 1;
}
__setup("ftrace_boot_snapshot", boot_snapshot);
static char trace_boot_options_buf[MAX_TRACER_SIZE] __initdata;
static int __init set_trace_boot_options(char *str)
@@ -7726,7 +7736,7 @@ const struct file_operations trace_min_max_fops = {
struct err_info {
const char **errs; /* ptr to loc-specific array of err strings */
u8 type; /* index into errs -> specific err string */
u8 pos; /* MAX_FILTER_STR_VAL = 256 */
u16 pos; /* caret position */
u64 ts;
};
@@ -7734,26 +7744,52 @@ struct tracing_log_err {
struct list_head list;
struct err_info info;
char loc[TRACING_LOG_LOC_MAX]; /* err location */
char cmd[MAX_FILTER_STR_VAL]; /* what caused err */
char *cmd; /* what caused err */
};
static DEFINE_MUTEX(tracing_err_log_lock);
static struct tracing_log_err *get_tracing_log_err(struct trace_array *tr)
static struct tracing_log_err *alloc_tracing_log_err(int len)
{
struct tracing_log_err *err;
err = kzalloc(sizeof(*err), GFP_KERNEL);
if (!err)
return ERR_PTR(-ENOMEM);
err->cmd = kzalloc(len, GFP_KERNEL);
if (!err->cmd) {
kfree(err);
return ERR_PTR(-ENOMEM);
}
return err;
}
static void free_tracing_log_err(struct tracing_log_err *err)
{
kfree(err->cmd);
kfree(err);
}
static struct tracing_log_err *get_tracing_log_err(struct trace_array *tr,
int len)
{
struct tracing_log_err *err;
if (tr->n_err_log_entries < TRACING_LOG_ERRS_MAX) {
err = kzalloc(sizeof(*err), GFP_KERNEL);
if (!err)
err = ERR_PTR(-ENOMEM);
else
err = alloc_tracing_log_err(len);
if (PTR_ERR(err) != -ENOMEM)
tr->n_err_log_entries++;
return err;
}
err = list_first_entry(&tr->err_log, struct tracing_log_err, list);
kfree(err->cmd);
err->cmd = kzalloc(len, GFP_KERNEL);
if (!err->cmd)
return ERR_PTR(-ENOMEM);
list_del(&err->list);
return err;
@@ -7814,22 +7850,25 @@ unsigned int err_pos(char *cmd, const char *str)
*/
void tracing_log_err(struct trace_array *tr,
const char *loc, const char *cmd,
const char **errs, u8 type, u8 pos)
const char **errs, u8 type, u16 pos)
{
struct tracing_log_err *err;
int len = 0;
if (!tr)
tr = &global_trace;
len += sizeof(CMD_PREFIX) + 2 * sizeof("\n") + strlen(cmd) + 1;
mutex_lock(&tracing_err_log_lock);
err = get_tracing_log_err(tr);
err = get_tracing_log_err(tr, len);
if (PTR_ERR(err) == -ENOMEM) {
mutex_unlock(&tracing_err_log_lock);
return;
}
snprintf(err->loc, TRACING_LOG_LOC_MAX, "%s: error: ", loc);
snprintf(err->cmd, MAX_FILTER_STR_VAL,"\n" CMD_PREFIX "%s\n", cmd);
snprintf(err->cmd, len, "\n" CMD_PREFIX "%s\n", cmd);
err->info.errs = errs;
err->info.type = type;
@@ -7847,7 +7886,7 @@ static void clear_tracing_err_log(struct trace_array *tr)
mutex_lock(&tracing_err_log_lock);
list_for_each_entry_safe(err, next, &tr->err_log, list) {
list_del(&err->list);
kfree(err);
free_tracing_log_err(err);
}
tr->n_err_log_entries = 0;
@@ -7875,9 +7914,9 @@ static void tracing_err_log_seq_stop(struct seq_file *m, void *v)
mutex_unlock(&tracing_err_log_lock);
}
static void tracing_err_log_show_pos(struct seq_file *m, u8 pos)
static void tracing_err_log_show_pos(struct seq_file *m, u16 pos)
{
u8 i;
u16 i;
for (i = 0; i < sizeof(CMD_PREFIX) - 1; i++)
seq_putc(m, ' ');
@@ -10155,6 +10194,14 @@ out:
return ret;
}
void __init ftrace_boot_snapshot(void)
{
if (snapshot_at_boot) {
tracing_snapshot();
internal_trace_puts("** Boot snapshot taken **\n");
}
}
void __init early_trace_init(void)
{
if (tracepoint_printk) {

2
kernel/trace/trace.h
View File

@@ -1877,7 +1877,7 @@ extern ssize_t trace_parse_run_command(struct file *file,
extern unsigned int err_pos(char *cmd, const char *str);
extern void tracing_log_err(struct trace_array *tr,
const char *loc, const char *cmd,
const char **errs, u8 type, u8 pos);
const char **errs, u8 type, u16 pos);
/*
* Normal trace_printk() and friends allocates special buffers

90
kernel/trace/trace_events.c
View File

@@ -40,6 +40,14 @@ static LIST_HEAD(ftrace_generic_fields);
static LIST_HEAD(ftrace_common_fields);
static bool eventdir_initialized;
static LIST_HEAD(module_strings);
struct module_string {
struct list_head next;
struct module *module;
char *str;
};
#define GFP_TRACE (GFP_KERNEL | __GFP_ZERO)
static struct kmem_cache *field_cachep;
@@ -2643,6 +2651,76 @@ static void update_event_printk(struct trace_event_call *call,
}
}
static void add_str_to_module(struct module *module, char *str)
{
struct module_string *modstr;
modstr = kmalloc(sizeof(*modstr), GFP_KERNEL);
/*
* If we failed to allocate memory here, then we'll just
* let the str memory leak when the module is removed.
* If this fails to allocate, there's worse problems than
* a leaked string on module removal.
*/
if (WARN_ON_ONCE(!modstr))
return;
modstr->module = module;
modstr->str = str;
list_add(&modstr->next, &module_strings);
}
static void update_event_fields(struct trace_event_call *call,
struct trace_eval_map *map)
{
struct ftrace_event_field *field;
struct list_head *head;
char *ptr;
char *str;
int len = strlen(map->eval_string);
/* Dynamic events should never have field maps */
if (WARN_ON_ONCE(call->flags & TRACE_EVENT_FL_DYNAMIC))
return;
head = trace_get_fields(call);
list_for_each_entry(field, head, link) {
ptr = strchr(field->type, '[');
if (!ptr)
continue;
ptr++;
if (!isalpha(*ptr) && *ptr != '_')
continue;
if (strncmp(map->eval_string, ptr, len) != 0)
continue;
str = kstrdup(field->type, GFP_KERNEL);
if (WARN_ON_ONCE(!str))
return;
ptr = str + (ptr - field->type);
ptr = eval_replace(ptr, map, len);
/* enum/sizeof string smaller than value */
if (WARN_ON_ONCE(!ptr)) {
kfree(str);
continue;
}
/*
* If the event is part of a module, then we need to free the string
* when the module is removed. Otherwise, it will stay allocated
* until a reboot.
*/
if (call->module)
add_str_to_module(call->module, str);
field->type = str;
}
}
void trace_event_eval_update(struct trace_eval_map **map, int len)
{
struct trace_event_call *call, *p;
@@ -2678,6 +2756,7 @@ void trace_event_eval_update(struct trace_eval_map **map, int len)
first = false;
}
update_event_printk(call, map[i]);
update_event_fields(call, map[i]);
}
}
}
@@ -2768,6 +2847,7 @@ int trace_add_event_call(struct trace_event_call *call)
mutex_unlock(&trace_types_lock);
return ret;
}
EXPORT_SYMBOL_GPL(trace_add_event_call);
/*
* Must be called under locking of trace_types_lock, event_mutex and
@@ -2829,6 +2909,7 @@ int trace_remove_event_call(struct trace_event_call *call)
return ret;
}
EXPORT_SYMBOL_GPL(trace_remove_event_call);
#define for_each_event(event, start, end) \
for (event = start; \
@@ -2863,6 +2944,7 @@ static void trace_module_add_events(struct module *mod)
static void trace_module_remove_events(struct module *mod)
{
struct trace_event_call *call, *p;
struct module_string *modstr, *m;
down_write(&trace_event_sem);
list_for_each_entry_safe(call, p, &ftrace_events, list) {
@@ -2871,6 +2953,14 @@ static void trace_module_remove_events(struct module *mod)
if (call->module == mod)
__trace_remove_event_call(call);
}
/* Check for any strings allocade for this module */
list_for_each_entry_safe(modstr, m, &module_strings, next) {
if (modstr->module != mod)
continue;
list_del(&modstr->next);
kfree(modstr->str);
kfree(modstr);
}
up_write(&trace_event_sem);
/*

33
kernel/trace/trace_events_hist.c
View File

@@ -727,11 +727,16 @@ static struct track_data *track_data_alloc(unsigned int key_len,
return data;
}
static char last_cmd[MAX_FILTER_STR_VAL];
#define HIST_PREFIX "hist:"
static char *last_cmd;
static char last_cmd_loc[MAX_FILTER_STR_VAL];
static int errpos(char *str)
{
if (!str || !last_cmd)
return 0;
return err_pos(last_cmd, str);
}
@@ -739,12 +744,22 @@ static void last_cmd_set(struct trace_event_file *file, char *str)
{
const char *system = NULL, *name = NULL;
struct trace_event_call *call;
int len;
if (!str)
return;
strcpy(last_cmd, "hist:");
strncat(last_cmd, str, MAX_FILTER_STR_VAL - 1 - sizeof("hist:"));
/* sizeof() contains the nul byte */
len = sizeof(HIST_PREFIX) + strlen(str);
kfree(last_cmd);
last_cmd = kzalloc(len, GFP_KERNEL);
if (!last_cmd)
return;
strcpy(last_cmd, HIST_PREFIX);
/* Again, sizeof() contains the nul byte */
len -= sizeof(HIST_PREFIX);
strncat(last_cmd, str, len);
if (file) {
call = file->event_call;
@@ -757,18 +772,22 @@ static void last_cmd_set(struct trace_event_file *file, char *str)
}
if (system)
snprintf(last_cmd_loc, MAX_FILTER_STR_VAL, "hist:%s:%s", system, name);
snprintf(last_cmd_loc, MAX_FILTER_STR_VAL, HIST_PREFIX "%s:%s", system, name);
}
static void hist_err(struct trace_array *tr, u8 err_type, u8 err_pos)
static void hist_err(struct trace_array *tr, u8 err_type, u16 err_pos)
{
if (!last_cmd)
return;
tracing_log_err(tr, last_cmd_loc, last_cmd, err_text,
err_type, err_pos);
}
static void hist_err_clear(void)
{
last_cmd[0] = '\0';
if (last_cmd)
last_cmd[0] = '\0';
last_cmd_loc[0] = '\0';
}
@@ -5610,7 +5629,7 @@ static int event_hist_trigger_print(struct seq_file *m,
bool have_var = false;
unsigned int i;
seq_puts(m, "hist:");
seq_puts(m, HIST_PREFIX);
if (data->name)
seq_printf(m, "%s:", data->name);

14
kernel/trace/trace_events_synth.c
View File

@@ -42,10 +42,13 @@ enum { ERRORS };
static const char *err_text[] = { ERRORS };
static char last_cmd[MAX_FILTER_STR_VAL];
static char *last_cmd;
static int errpos(const char *str)
{
if (!str || !last_cmd)
return 0;
return err_pos(last_cmd, str);
}
@@ -54,11 +57,16 @@ static void last_cmd_set(const char *str)
if (!str)
return;
strncpy(last_cmd, str, MAX_FILTER_STR_VAL - 1);
kfree(last_cmd);
last_cmd = kstrdup(str, GFP_KERNEL);
}
static void synth_err(u8 err_type, u8 err_pos)
static void synth_err(u8 err_type, u16 err_pos)
{
if (!last_cmd)
return;
tracing_log_err(NULL, "synthetic_events", last_cmd, err_text,
err_type, err_pos);
}

1690
kernel/trace/trace_events_user.c Normal file
View File

File diff suppressed because it is too large Load Diff

2
kernel/trace/trace_sched_switch.c
View File

@@ -45,7 +45,7 @@ probe_sched_wakeup(void *ignore, struct task_struct *wakee)
if (!flags)
return;
tracing_record_taskinfo(current, flags);
tracing_record_taskinfo_sched_switch(current, wakee, flags);
}
static int tracing_sched_register(void)

49
lib/vsprintf.c
View File

@@ -49,10 +49,15 @@
#include <asm/page.h> /* for PAGE_SIZE */
#include <asm/byteorder.h> /* cpu_to_le16 */
#include <asm/unaligned.h>
#include <linux/string_helpers.h>
#include "kstrtox.h"
/* Disable pointer hashing if requested */
bool no_hash_pointers __ro_after_init;
EXPORT_SYMBOL_GPL(no_hash_pointers);
static noinline unsigned long long simple_strntoull(const char *startp, size_t max_chars, char **endp, unsigned int base)
{
const char *cp;
@@ -850,6 +855,19 @@ static char *ptr_to_id(char *buf, char *end, const void *ptr,
return pointer_string(buf, end, (const void *)hashval, spec);
}
static char *default_pointer(char *buf, char *end, const void *ptr,
struct printf_spec spec)
{
/*
* default is to _not_ leak addresses, so hash before printing,
* unless no_hash_pointers is specified on the command line.
*/
if (unlikely(no_hash_pointers))
return pointer_string(buf, end, ptr, spec);
return ptr_to_id(buf, end, ptr, spec);
}
int kptr_restrict __read_mostly;
static noinline_for_stack
@@ -859,7 +877,7 @@ char *restricted_pointer(char *buf, char *end, const void *ptr,
switch (kptr_restrict) {
case 0:
/* Handle as %p, hash and do _not_ leak addresses. */
return ptr_to_id(buf, end, ptr, spec);
return default_pointer(buf, end, ptr, spec);
case 1: {
const struct cred *cred;
@@ -1763,7 +1781,7 @@ char *fourcc_string(char *buf, char *end, const u32 *fourcc,
char output[sizeof("0123 little-endian (0x01234567)")];
char *p = output;
unsigned int i;
u32 val;
u32 orig, val;
if (fmt[1] != 'c' || fmt[2] != 'c')
return error_string(buf, end, "(%p4?)", spec);
@@ -1771,21 +1789,23 @@ char *fourcc_string(char *buf, char *end, const u32 *fourcc,
if (check_pointer(&buf, end, fourcc, spec))
return buf;
val = *fourcc & ~BIT(31);
orig = get_unaligned(fourcc);
val = orig & ~BIT(31);
for (i = 0; i < sizeof(*fourcc); i++) {
for (i = 0; i < sizeof(u32); i++) {
unsigned char c = val >> (i * 8);
/* Print non-control ASCII characters as-is, dot otherwise */
*p++ = isascii(c) && isprint(c) ? c : '.';
}
strcpy(p, *fourcc & BIT(31) ? " big-endian" : " little-endian");
*p++ = ' ';
strcpy(p, orig & BIT(31) ? "big-endian" : "little-endian");
p += strlen(p);
*p++ = ' ';
*p++ = '(';
p = special_hex_number(p, output + sizeof(output) - 2, *fourcc, sizeof(u32));
p = special_hex_number(p, output + sizeof(output) - 2, orig, sizeof(u32));
*p++ = ')';
*p = '\0';
@@ -2225,10 +2245,6 @@ char *fwnode_string(char *buf, char *end, struct fwnode_handle *fwnode,
return widen_string(buf, buf - buf_start, end, spec);
}
/* Disable pointer hashing if requested */
bool no_hash_pointers __ro_after_init;
EXPORT_SYMBOL_GPL(no_hash_pointers);
int __init no_hash_pointers_enable(char *str)
{
if (no_hash_pointers)
@@ -2457,7 +2473,7 @@ char *pointer(const char *fmt, char *buf, char *end, void *ptr,
case 'e':
/* %pe with a non-ERR_PTR gets treated as plain %p */
if (!IS_ERR(ptr))
break;
return default_pointer(buf, end, ptr, spec);
return err_ptr(buf, end, ptr, spec);
case 'u':
case 'k':
@@ -2467,16 +2483,9 @@ char *pointer(const char *fmt, char *buf, char *end, void *ptr,
default:
return error_string(buf, end, "(einval)", spec);
}
default:
return default_pointer(buf, end, ptr, spec);
}
/*
* default is to _not_ leak addresses, so hash before printing,
* unless no_hash_pointers is specified on the command line.
*/
if (unlikely(no_hash_pointers))
return pointer_string(buf, end, ptr, spec);
else
return ptr_to_id(buf, end, ptr, spec);
}
/*

2
mm/slab_common.c
View File

@@ -807,7 +807,7 @@ void __init setup_kmalloc_cache_index_table(void)
unsigned int i;
BUILD_BUG_ON(KMALLOC_MIN_SIZE > 256 ||
(KMALLOC_MIN_SIZE & (KMALLOC_MIN_SIZE - 1)));
!is_power_of_2(KMALLOC_MIN_SIZE));
for (i = 8; i < KMALLOC_MIN_SIZE; i += 8) {
unsigned int elem = size_index_elem(i);

2
mm/slob.c
View File

@@ -714,7 +714,7 @@ int __kmem_cache_shrink(struct kmem_cache *d)
return 0;
}
struct kmem_cache kmem_cache_boot = {
static struct kmem_cache kmem_cache_boot = {
.name = "kmem_cache",
.size = sizeof(struct kmem_cache),
.flags = SLAB_PANIC,

130
mm/slub.c
View File

@@ -1788,8 +1788,8 @@ static void *setup_object(struct kmem_cache *s, struct slab *slab,
/*
* Slab allocation and freeing
*/
static inline struct slab *alloc_slab_page(struct kmem_cache *s,
gfp_t flags, int node, struct kmem_cache_order_objects oo)
static inline struct slab *alloc_slab_page(gfp_t flags, int node,
struct kmem_cache_order_objects oo)
{
struct folio *folio;
struct slab *slab;
@@ -1941,7 +1941,7 @@ static struct slab *allocate_slab(struct kmem_cache *s, gfp_t flags, int node)
if ((alloc_gfp & __GFP_DIRECT_RECLAIM) && oo_order(oo) > oo_order(s->min))
alloc_gfp = (alloc_gfp | __GFP_NOMEMALLOC) & ~(__GFP_RECLAIM|__GFP_NOFAIL);
slab = alloc_slab_page(s, alloc_gfp, node, oo);
slab = alloc_slab_page(alloc_gfp, node, oo);
if (unlikely(!slab)) {
oo = s->min;
alloc_gfp = flags;
@@ -1949,7 +1949,7 @@ static struct slab *allocate_slab(struct kmem_cache *s, gfp_t flags, int node)
* Allocation may have failed due to fragmentation.
* Try a lower order alloc if possible
*/
slab = alloc_slab_page(s, alloc_gfp, node, oo);
slab = alloc_slab_page(alloc_gfp, node, oo);
if (unlikely(!slab))
goto out;
stat(s, ORDER_FALLBACK);
@@ -2348,10 +2348,10 @@ static void init_kmem_cache_cpus(struct kmem_cache *s)
static void deactivate_slab(struct kmem_cache *s, struct slab *slab,
void *freelist)
{
enum slab_modes { M_NONE, M_PARTIAL, M_FULL, M_FREE };
enum slab_modes { M_NONE, M_PARTIAL, M_FULL, M_FREE, M_FULL_NOLIST };
struct kmem_cache_node *n = get_node(s, slab_nid(slab));
int lock = 0, free_delta = 0;
enum slab_modes l = M_NONE, m = M_NONE;
int free_delta = 0;
enum slab_modes mode = M_NONE;
void *nextfree, *freelist_iter, *freelist_tail;
int tail = DEACTIVATE_TO_HEAD;
unsigned long flags = 0;
@@ -2393,14 +2393,10 @@ static void deactivate_slab(struct kmem_cache *s, struct slab *slab,
* Ensure that the slab is unfrozen while the list presence
* reflects the actual number of objects during unfreeze.
*
* We setup the list membership and then perform a cmpxchg
* with the count. If there is a mismatch then the slab
* is not unfrozen but the slab is on the wrong list.
*
* Then we restart the process which may have to remove
* the slab from the list that we just put it on again
* because the number of objects in the slab may have
* changed.
* We first perform cmpxchg holding lock and insert to list
* when it succeed. If there is mismatch then the slab is not
* unfrozen and number of objects in the slab may have changed.
* Then release lock and retry cmpxchg again.
*/
redo:
@@ -2419,61 +2415,52 @@ redo:
new.frozen = 0;
if (!new.inuse && n->nr_partial >= s->min_partial)
m = M_FREE;
else if (new.freelist) {
m = M_PARTIAL;
if (!lock) {
lock = 1;
/*
* Taking the spinlock removes the possibility that
* acquire_slab() will see a slab that is frozen
*/
spin_lock_irqsave(&n->list_lock, flags);
}
if (!new.inuse && n->nr_partial >= s->min_partial) {
mode = M_FREE;
} else if (new.freelist) {
mode = M_PARTIAL;
/*
* Taking the spinlock removes the possibility that
* acquire_slab() will see a slab that is frozen
*/
spin_lock_irqsave(&n->list_lock, flags);
} else if (kmem_cache_debug_flags(s, SLAB_STORE_USER)) {
mode = M_FULL;
/*
* This also ensures that the scanning of full
* slabs from diagnostic functions will not see
* any frozen slabs.
*/
spin_lock_irqsave(&n->list_lock, flags);
} else {
m = M_FULL;
if (kmem_cache_debug_flags(s, SLAB_STORE_USER) && !lock) {
lock = 1;
/*
* This also ensures that the scanning of full
* slabs from diagnostic functions will not see
* any frozen slabs.
*/
spin_lock_irqsave(&n->list_lock, flags);
}
mode = M_FULL_NOLIST;
}
if (l != m) {
if (l == M_PARTIAL)
remove_partial(n, slab);
else if (l == M_FULL)
remove_full(s, n, slab);
if (m == M_PARTIAL)
add_partial(n, slab, tail);
else if (m == M_FULL)
add_full(s, n, slab);
}
l = m;
if (!cmpxchg_double_slab(s, slab,
old.freelist, old.counters,
new.freelist, new.counters,
"unfreezing slab"))
"unfreezing slab")) {
if (mode == M_PARTIAL || mode == M_FULL)
spin_unlock_irqrestore(&n->list_lock, flags);
goto redo;
}
if (lock)
if (mode == M_PARTIAL) {
add_partial(n, slab, tail);
spin_unlock_irqrestore(&n->list_lock, flags);
if (m == M_PARTIAL)
stat(s, tail);
else if (m == M_FULL)
stat(s, DEACTIVATE_FULL);
else if (m == M_FREE) {
} else if (mode == M_FREE) {
stat(s, DEACTIVATE_EMPTY);
discard_slab(s, slab);
stat(s, FREE_SLAB);
} else if (mode == M_FULL) {
add_full(s, n, slab);
spin_unlock_irqrestore(&n->list_lock, flags);
stat(s, DEACTIVATE_FULL);
} else if (mode == M_FULL_NOLIST) {
stat(s, DEACTIVATE_FULL);
}
}
@@ -4014,15 +4001,6 @@ static int init_kmem_cache_nodes(struct kmem_cache *s)
return 1;
}
static void set_min_partial(struct kmem_cache *s, unsigned long min)
{
if (min < MIN_PARTIAL)
min = MIN_PARTIAL;
else if (min > MAX_PARTIAL)
min = MAX_PARTIAL;
s->min_partial = min;
}
static void set_cpu_partial(struct kmem_cache *s)
{
#ifdef CONFIG_SLUB_CPU_PARTIAL
@@ -4060,7 +4038,7 @@ static void set_cpu_partial(struct kmem_cache *s)
* calculate_sizes() determines the order and the distribution of data within
* a slab object.
*/
static int calculate_sizes(struct kmem_cache *s, int forced_order)
static int calculate_sizes(struct kmem_cache *s)
{
slab_flags_t flags = s->flags;
unsigned int size = s->object_size;
@@ -4164,10 +4142,7 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)
size = ALIGN(size, s->align);
s->size = size;
s->reciprocal_size = reciprocal_value(size);
if (forced_order >= 0)
order = forced_order;
else
order = calculate_order(size);
order = calculate_order(size);
if ((int)order < 0)
return 0;
@@ -4203,7 +4178,7 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags)
s->random = get_random_long();
#endif
if (!calculate_sizes(s, -1))
if (!calculate_sizes(s))
goto error;
if (disable_higher_order_debug) {
/*
@@ -4213,7 +4188,7 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags)
if (get_order(s->size) > get_order(s->object_size)) {
s->flags &= ~DEBUG_METADATA_FLAGS;
s->offset = 0;
if (!calculate_sizes(s, -1))
if (!calculate_sizes(s))
goto error;
}
}
@@ -4229,7 +4204,8 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags)
* The larger the object size is, the more slabs we want on the partial
* list to avoid pounding the page allocator excessively.
*/
set_min_partial(s, ilog2(s->size) / 2);
s->min_partial = min_t(unsigned long, MAX_PARTIAL, ilog2(s->size) / 2);
s->min_partial = max_t(unsigned long, MIN_PARTIAL, s->min_partial);
set_cpu_partial(s);
@@ -5358,12 +5334,10 @@ struct slab_attribute {
};
#define SLAB_ATTR_RO(_name) \
static struct slab_attribute _name##_attr = \
__ATTR(_name, 0400, _name##_show, NULL)
static struct slab_attribute _name##_attr = __ATTR_RO_MODE(_name, 0400)
#define SLAB_ATTR(_name) \
static struct slab_attribute _name##_attr = \
__ATTR(_name, 0600, _name##_show, _name##_store)
static struct slab_attribute _name##_attr = __ATTR_RW_MODE(_name, 0600)
static ssize_t slab_size_show(struct kmem_cache *s, char *buf)
{
@@ -5410,7 +5384,7 @@ static ssize_t min_partial_store(struct kmem_cache *s, const char *buf,
if (err)
return err;
set_min_partial(s, min);
s->min_partial = min;
return length;
}
SLAB_ATTR(min_partial);

8
samples/Kconfig
View File

@@ -14,7 +14,13 @@ config SAMPLE_TRACE_EVENTS
tristate "Build trace_events examples -- loadable modules only"
depends on EVENT_TRACING && m
help
This build trace event example modules.
This builds the trace event example module.
config SAMPLE_TRACE_CUSTOM_EVENTS
tristate "Build custom trace event example -- loadable modules only"
depends on EVENT_TRACING && m
help
This builds the custom trace event example module.
config SAMPLE_TRACE_PRINTK
tristate "Build trace_printk module - tests various trace_printk formats"

1
samples/Makefile
View File

@@ -20,6 +20,7 @@ obj-$(CONFIG_SAMPLE_RPMSG_CLIENT) += rpmsg/
subdir-$(CONFIG_SAMPLE_SECCOMP) += seccomp
subdir-$(CONFIG_SAMPLE_TIMER) += timers
obj-$(CONFIG_SAMPLE_TRACE_EVENTS) += trace_events/
obj-$(CONFIG_SAMPLE_TRACE_CUSTOM_EVENTS) += trace_events/
obj-$(CONFIG_SAMPLE_TRACE_PRINTK) += trace_printk/
obj-$(CONFIG_SAMPLE_FTRACE_DIRECT) += ftrace/
obj-$(CONFIG_SAMPLE_FTRACE_DIRECT_MULTI) += ftrace/

2
samples/trace_events/Makefile
View File

@@ -11,5 +11,7 @@
# Here trace-events-sample.c does the CREATE_TRACE_POINTS.
#
CFLAGS_trace-events-sample.o := -I$(src)
CFLAGS_trace_custom_sched.o := -I$(src)
obj-$(CONFIG_SAMPLE_TRACE_EVENTS) += trace-events-sample.o
obj-$(CONFIG_SAMPLE_TRACE_CUSTOM_EVENTS) += trace_custom_sched.o

60
samples/trace_events/trace_custom_sched.c Normal file
View File

@@ -0,0 +1,60 @@
// SPDX-License-Identifier: GPL-2.0
/*
* event tracer
*
* Copyright (C) 2022 Google Inc, Steven Rostedt <rostedt@goodmis.org>
*/
#define pr_fmt(fmt) fmt
#include <linux/trace_events.h>
#include <linux/version.h>
#include <linux/module.h>
#include <linux/sched.h>
/*
* Must include the event header that the custom event will attach to,
* from the C file, and not in the custom header file.
*/
#include <trace/events/sched.h>
/* Declare CREATE_CUSTOM_TRACE_EVENTS before including custom header */
#define CREATE_CUSTOM_TRACE_EVENTS
#include "trace_custom_sched.h"
/*
* As the trace events are not exported to modules, the use of
* for_each_kernel_tracepoint() is needed to find the trace event
* to attach to. The fct() function below, is a callback that
* will be called for every event.
*
* Helper functions are created by the TRACE_CUSTOM_EVENT() macro
* update the event. Those are of the form:
*
* trace_custom_event_<event>_update()
*
* Where <event> is the event to attach.
*/
static void fct(struct tracepoint *tp, void *priv)
{
trace_custom_event_sched_switch_update(tp);
trace_custom_event_sched_waking_update(tp);
}
static int __init trace_sched_init(void)
{
for_each_kernel_tracepoint(fct, NULL);
return 0;
}
static void __exit trace_sched_exit(void)
{
}
module_init(trace_sched_init);
module_exit(trace_sched_exit);
MODULE_AUTHOR("Steven Rostedt");
MODULE_DESCRIPTION("Custom scheduling events");
MODULE_LICENSE("GPL");

96
samples/trace_events/trace_custom_sched.h Normal file
View File

@@ -0,0 +1,96 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Like the headers that use TRACE_EVENT(), the TRACE_CUSTOM_EVENT()
* needs a header that allows for multiple inclusions.
*
* Test for a unique name (here we have _TRACE_CUSTOM_SCHED_H),
* also allowing to continue if TRACE_CUSTOM_MULTI_READ is defined.
*/
#if !defined(_TRACE_CUSTOM_SCHED_H) || defined(TRACE_CUSTOM_MULTI_READ)
#define _TRACE_CUSTOM_SCHED_H
/* Include linux/trace_events.h for initial defines of TRACE_CUSTOM_EVENT() */
#include <linux/trace_events.h>
/*
* TRACE_CUSTOM_EVENT() is just like TRACE_EVENT(). The first parameter
* is the event name of an existing event where the TRACE_EVENT has been included
* in the C file before including this file.
*/
TRACE_CUSTOM_EVENT(sched_switch,
/*
* The TP_PROTO() and TP_ARGS must match the trace event
* that the custom event is using.
*/
TP_PROTO(bool preempt,
unsigned int prev_state,
struct task_struct *prev,
struct task_struct *next),
TP_ARGS(preempt, prev_state, prev, next),
/*
* The next fields are where the customization happens.
* The TP_STRUCT__entry() defines what will be recorded
* in the ring buffer when the custom event triggers.
*
* The rest is just like the TRACE_EVENT() macro except that
* it uses the custom entry.
*/
TP_STRUCT__entry(
__field( unsigned short, prev_prio )
__field( unsigned short, next_prio )
__field( pid_t, next_pid )
),
TP_fast_assign(
__entry->prev_prio = prev->prio;
__entry->next_pid = next->pid;
__entry->next_prio = next->prio;
),
TP_printk("prev_prio=%d next_pid=%d next_prio=%d",
__entry->prev_prio, __entry->next_pid, __entry->next_prio)
)
TRACE_CUSTOM_EVENT(sched_waking,
TP_PROTO(struct task_struct *p),
TP_ARGS(p),
TP_STRUCT__entry(
__field( pid_t, pid )
__field( unsigned short, prio )
),
TP_fast_assign(
__entry->pid = p->pid;
__entry->prio = p->prio;
),
TP_printk("pid=%d prio=%d", __entry->pid, __entry->prio)
)
#endif
/*
* Just like the headers that create TRACE_EVENTs, the below must
* be outside the protection of the above #if block.
*/
/*
* It is required that the Makefile includes:
* CFLAGS_<c_file>.o := -I$(src)
*/
#undef TRACE_INCLUDE_PATH
#undef TRACE_INCLUDE_FILE
#define TRACE_INCLUDE_PATH .
/*
* It is requred that the TRACE_INCLUDE_FILE be the same
* as this file without the ".h".
*/
#define TRACE_INCLUDE_FILE trace_custom_sched
#include <trace/define_custom_trace.h>

5
samples/user_events/Makefile Normal file
View File

@@ -0,0 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
CFLAGS += -Wl,-no-as-needed -Wall -I../../usr/include
example: example.o
example.o: example.c

91
samples/user_events/example.c Normal file
View File

@@ -0,0 +1,91 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright (c) 2021, Microsoft Corporation.
*
* Authors:
* Beau Belgrave <beaub@linux.microsoft.com>
*/
#include <errno.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <linux/user_events.h>
/* Assumes debugfs is mounted */
const char *data_file = "/sys/kernel/debug/tracing/user_events_data";
const char *status_file = "/sys/kernel/debug/tracing/user_events_status";
static int event_status(char **status)
{
int fd = open(status_file, O_RDONLY);
*status = mmap(NULL, sysconf(_SC_PAGESIZE), PROT_READ,
MAP_SHARED, fd, 0);
close(fd);
if (*status == MAP_FAILED)
return -1;
return 0;
}
static int event_reg(int fd, const char *command, int *status, int *write)
{
struct user_reg reg = {0};
reg.size = sizeof(reg);
reg.name_args = (__u64)command;
if (ioctl(fd, DIAG_IOCSREG, &reg) == -1)
return -1;
*status = reg.status_index;
*write = reg.write_index;
return 0;
}
int main(int argc, char **argv)
{
int data_fd, status, write;
char *status_page;
struct iovec io[2];
__u32 count = 0;
if (event_status(&status_page) == -1)
return errno;
data_fd = open(data_file, O_RDWR);
if (event_reg(data_fd, "test u32 count", &status, &write) == -1)
return errno;
/* Setup iovec */
io[0].iov_base = &write;
io[0].iov_len = sizeof(write);
io[1].iov_base = &count;
io[1].iov_len = sizeof(count);
ask:
printf("Press enter to check status...\n");
getchar();
/* Check if anyone is listening */
if (status_page[status]) {
/* Yep, trace out our data */
writev(data_fd, (const struct iovec *)io, 2);
/* Increase the count */
count++;
printf("Something was attached, wrote data\n");
}
goto ask;
return 0;
}

9
tools/testing/selftests/user_events/Makefile Normal file
View File

@@ -0,0 +1,9 @@
# SPDX-License-Identifier: GPL-2.0
CFLAGS += -Wl,-no-as-needed -Wall -I../../../../usr/include
LDLIBS += -lrt -lpthread -lm
TEST_GEN_PROGS = ftrace_test dyn_test perf_test
TEST_FILES := settings
include ../lib.mk

130
tools/testing/selftests/user_events/dyn_test.c Normal file
View File

@@ -0,0 +1,130 @@
// SPDX-License-Identifier: GPL-2.0
/*
* User Events Dyn Events Test Program
*
* Copyright (c) 2021 Beau Belgrave <beaub@linux.microsoft.com>
*/
#include <errno.h>
#include <linux/user_events.h>
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <sys/stat.h>
#include <unistd.h>
#include "../kselftest_harness.h"
const char *dyn_file = "/sys/kernel/debug/tracing/dynamic_events";
const char *clear = "!u:__test_event";
static int Append(const char *value)
{
int fd = open(dyn_file, O_RDWR | O_APPEND);
int ret = write(fd, value, strlen(value));
close(fd);
return ret;
}
#define CLEAR() \
do { \
int ret = Append(clear); \
if (ret == -1) \
ASSERT_EQ(ENOENT, errno); \
} while (0)
#define TEST_PARSE(x) \
do { \
ASSERT_NE(-1, Append(x)); \
CLEAR(); \
} while (0)
#define TEST_NPARSE(x) ASSERT_EQ(-1, Append(x))
FIXTURE(user) {
};
FIXTURE_SETUP(user) {
CLEAR();
}
FIXTURE_TEARDOWN(user) {
CLEAR();
}
TEST_F(user, basic_types) {
/* All should work */
TEST_PARSE("u:__test_event u64 a");
TEST_PARSE("u:__test_event u32 a");
TEST_PARSE("u:__test_event u16 a");
TEST_PARSE("u:__test_event u8 a");
TEST_PARSE("u:__test_event char a");
TEST_PARSE("u:__test_event unsigned char a");
TEST_PARSE("u:__test_event int a");
TEST_PARSE("u:__test_event unsigned int a");
TEST_PARSE("u:__test_event short a");
TEST_PARSE("u:__test_event unsigned short a");
TEST_PARSE("u:__test_event char[20] a");
TEST_PARSE("u:__test_event unsigned char[20] a");
TEST_PARSE("u:__test_event char[0x14] a");
TEST_PARSE("u:__test_event unsigned char[0x14] a");
/* Bad size format should fail */
TEST_NPARSE("u:__test_event char[aa] a");
/* Large size should fail */
TEST_NPARSE("u:__test_event char[9999] a");
/* Long size string should fail */
TEST_NPARSE("u:__test_event char[0x0000000000001] a");
}
TEST_F(user, loc_types) {
/* All should work */
TEST_PARSE("u:__test_event __data_loc char[] a");
TEST_PARSE("u:__test_event __data_loc unsigned char[] a");
TEST_PARSE("u:__test_event __rel_loc char[] a");
TEST_PARSE("u:__test_event __rel_loc unsigned char[] a");
}
TEST_F(user, size_types) {
/* Should work */
TEST_PARSE("u:__test_event struct custom a 20");
/* Size not specified on struct should fail */
TEST_NPARSE("u:__test_event struct custom a");
/* Size specified on non-struct should fail */
TEST_NPARSE("u:__test_event char a 20");
}
TEST_F(user, flags) {
/* Should work */
TEST_PARSE("u:__test_event:BPF_ITER u32 a");
/* Forward compat */
TEST_PARSE("u:__test_event:BPF_ITER,FLAG_FUTURE u32 a");
}
TEST_F(user, matching) {
/* Register */
ASSERT_NE(-1, Append("u:__test_event struct custom a 20"));
/* Should not match */
TEST_NPARSE("!u:__test_event struct custom b");
/* Should match */
TEST_PARSE("!u:__test_event struct custom a");
/* Multi field reg */
ASSERT_NE(-1, Append("u:__test_event u32 a; u32 b"));
/* Non matching cases */
TEST_NPARSE("!u:__test_event u32 a");
TEST_NPARSE("!u:__test_event u32 b");
TEST_NPARSE("!u:__test_event u32 a; u32 ");
TEST_NPARSE("!u:__test_event u32 a; u32 a");
/* Matching case */
TEST_PARSE("!u:__test_event u32 a; u32 b");
/* Register */
ASSERT_NE(-1, Append("u:__test_event u32 a; u32 b"));
/* Ensure trailing semi-colon case */
TEST_PARSE("!u:__test_event u32 a; u32 b;");
}
int main(int argc, char **argv)
{
return test_harness_run(argc, argv);
}

452
tools/testing/selftests/user_events/ftrace_test.c Normal file
View File

@@ -0,0 +1,452 @@
// SPDX-License-Identifier: GPL-2.0
/*
* User Events FTrace Test Program
*
* Copyright (c) 2021 Beau Belgrave <beaub@linux.microsoft.com>
*/
#include <errno.h>
#include <linux/user_events.h>
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <sys/stat.h>
#include <unistd.h>
#include "../kselftest_harness.h"
const char *data_file = "/sys/kernel/debug/tracing/user_events_data";
const char *status_file = "/sys/kernel/debug/tracing/user_events_status";
const char *enable_file = "/sys/kernel/debug/tracing/events/user_events/__test_event/enable";
const char *trace_file = "/sys/kernel/debug/tracing/trace";
const char *fmt_file = "/sys/kernel/debug/tracing/events/user_events/__test_event/format";
static int trace_bytes(void)
{
int fd = open(trace_file, O_RDONLY);
char buf[256];
int bytes = 0, got;
if (fd == -1)
return -1;
while (true) {
got = read(fd, buf, sizeof(buf));
if (got == -1)
return -1;
if (got == 0)
break;
bytes += got;
}
close(fd);
return bytes;
}
static int skip_until_empty_line(FILE *fp)
{
int c, last = 0;
while (true) {
c = getc(fp);
if (c == EOF)
break;
if (last == '\n' && c == '\n')
return 0;
last = c;
}
return -1;
}
static int get_print_fmt(char *buffer, int len)
{
FILE *fp = fopen(fmt_file, "r");
char *newline;
if (!fp)
return -1;
/* Read until empty line (Skip Common) */
if (skip_until_empty_line(fp) < 0)
goto err;
/* Read until empty line (Skip Properties) */
if (skip_until_empty_line(fp) < 0)
goto err;
/* Read in print_fmt: */
if (fgets(buffer, len, fp) == NULL)
goto err;
newline = strchr(buffer, '\n');
if (newline)
*newline = '\0';
fclose(fp);
return 0;
err:
fclose(fp);
return -1;
}
static int clear(void)
{
int fd = open(data_file, O_RDWR);
if (fd == -1)
return -1;
if (ioctl(fd, DIAG_IOCSDEL, "__test_event") == -1)
if (errno != ENOENT)
return -1;
close(fd);
return 0;
}
static int check_print_fmt(const char *event, const char *expected)
{
struct user_reg reg = {0};
char print_fmt[256];
int ret;
int fd;
/* Ensure cleared */
ret = clear();
if (ret != 0)
return ret;
fd = open(data_file, O_RDWR);
if (fd == -1)
return fd;
reg.size = sizeof(reg);
reg.name_args = (__u64)event;
/* Register should work */
ret = ioctl(fd, DIAG_IOCSREG, &reg);
close(fd);
if (ret != 0)
return ret;
/* Ensure correct print_fmt */
ret = get_print_fmt(print_fmt, sizeof(print_fmt));
if (ret != 0)
return ret;
return strcmp(print_fmt, expected);
}
FIXTURE(user) {
int status_fd;
int data_fd;
int enable_fd;
};
FIXTURE_SETUP(user) {
self->status_fd = open(status_file, O_RDONLY);
ASSERT_NE(-1, self->status_fd);
self->data_fd = open(data_file, O_RDWR);
ASSERT_NE(-1, self->data_fd);
self->enable_fd = -1;
}
FIXTURE_TEARDOWN(user) {
close(self->status_fd);
close(self->data_fd);
if (self->enable_fd != -1) {
write(self->enable_fd, "0", sizeof("0"));
close(self->enable_fd);
}
ASSERT_EQ(0, clear());
}
TEST_F(user, register_events) {
struct user_reg reg = {0};
int page_size = sysconf(_SC_PAGESIZE);
char *status_page;
reg.size = sizeof(reg);
reg.name_args = (__u64)"__test_event u32 field1; u32 field2";
status_page = mmap(NULL, page_size, PROT_READ, MAP_SHARED,
self->status_fd, 0);
/* Register should work */
ASSERT_EQ(0, ioctl(self->data_fd, DIAG_IOCSREG, &reg));
ASSERT_EQ(0, reg.write_index);
ASSERT_NE(0, reg.status_index);
/* Multiple registers should result in same index */
ASSERT_EQ(0, ioctl(self->data_fd, DIAG_IOCSREG, &reg));
ASSERT_EQ(0, reg.write_index);
ASSERT_NE(0, reg.status_index);
/* Ensure disabled */
self->enable_fd = open(enable_file, O_RDWR);
ASSERT_NE(-1, self->enable_fd);
ASSERT_NE(-1, write(self->enable_fd, "0", sizeof("0")))
/* MMAP should work and be zero'd */
ASSERT_NE(MAP_FAILED, status_page);
ASSERT_NE(NULL, status_page);
ASSERT_EQ(0, status_page[reg.status_index]);
/* Enable event and ensure bits updated in status */
ASSERT_NE(-1, write(self->enable_fd, "1", sizeof("1")))
ASSERT_EQ(EVENT_STATUS_FTRACE, status_page[reg.status_index]);
/* Disable event and ensure bits updated in status */
ASSERT_NE(-1, write(self->enable_fd, "0", sizeof("0")))
ASSERT_EQ(0, status_page[reg.status_index]);
/* File still open should return -EBUSY for delete */
ASSERT_EQ(-1, ioctl(self->data_fd, DIAG_IOCSDEL, "__test_event"));
ASSERT_EQ(EBUSY, errno);
/* Delete should work only after close */
close(self->data_fd);
self->data_fd = open(data_file, O_RDWR);
ASSERT_EQ(0, ioctl(self->data_fd, DIAG_IOCSDEL, "__test_event"));
/* Unmap should work */
ASSERT_EQ(0, munmap(status_page, page_size));
}
TEST_F(user, write_events) {
struct user_reg reg = {0};
struct iovec io[3];
__u32 field1, field2;
int before = 0, after = 0;
reg.size = sizeof(reg);
reg.name_args = (__u64)"__test_event u32 field1; u32 field2";
field1 = 1;
field2 = 2;
io[0].iov_base = &reg.write_index;
io[0].iov_len = sizeof(reg.write_index);
io[1].iov_base = &field1;
io[1].iov_len = sizeof(field1);
io[2].iov_base = &field2;
io[2].iov_len = sizeof(field2);
/* Register should work */
ASSERT_EQ(0, ioctl(self->data_fd, DIAG_IOCSREG, &reg));
ASSERT_EQ(0, reg.write_index);
ASSERT_NE(0, reg.status_index);
/* Write should fail on invalid slot with ENOENT */
io[0].iov_base = &field2;
io[0].iov_len = sizeof(field2);
ASSERT_EQ(-1, writev(self->data_fd, (const struct iovec *)io, 3));
ASSERT_EQ(ENOENT, errno);
io[0].iov_base = &reg.write_index;
io[0].iov_len = sizeof(reg.write_index);
/* Enable event */
self->enable_fd = open(enable_file, O_RDWR);
ASSERT_NE(-1, write(self->enable_fd, "1", sizeof("1")))
/* Write should make it out to ftrace buffers */
before = trace_bytes();
ASSERT_NE(-1, writev(self->data_fd, (const struct iovec *)io, 3));
after = trace_bytes();
ASSERT_GT(after, before);
}
TEST_F(user, write_fault) {
struct user_reg reg = {0};
struct iovec io[2];
int l = sizeof(__u64);
void *anon;
reg.size = sizeof(reg);
reg.name_args = (__u64)"__test_event u64 anon";
anon = mmap(NULL, l, PROT_READ, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
ASSERT_NE(MAP_FAILED, anon);
io[0].iov_base = &reg.write_index;
io[0].iov_len = sizeof(reg.write_index);
io[1].iov_base = anon;
io[1].iov_len = l;
/* Register should work */
ASSERT_EQ(0, ioctl(self->data_fd, DIAG_IOCSREG, &reg));
ASSERT_EQ(0, reg.write_index);
ASSERT_NE(0, reg.status_index);
/* Write should work normally */
ASSERT_NE(-1, writev(self->data_fd, (const struct iovec *)io, 2));
/* Faulted data should zero fill and work */
ASSERT_EQ(0, madvise(anon, l, MADV_DONTNEED));
ASSERT_NE(-1, writev(self->data_fd, (const struct iovec *)io, 2));
ASSERT_EQ(0, munmap(anon, l));
}
TEST_F(user, write_validator) {
struct user_reg reg = {0};
struct iovec io[3];
int loc, bytes;
char data[8];
int before = 0, after = 0;
reg.size = sizeof(reg);
reg.name_args = (__u64)"__test_event __rel_loc char[] data";
/* Register should work */
ASSERT_EQ(0, ioctl(self->data_fd, DIAG_IOCSREG, &reg));
ASSERT_EQ(0, reg.write_index);
ASSERT_NE(0, reg.status_index);
io[0].iov_base = &reg.write_index;
io[0].iov_len = sizeof(reg.write_index);
io[1].iov_base = &loc;
io[1].iov_len = sizeof(loc);
io[2].iov_base = data;
bytes = snprintf(data, sizeof(data), "Test") + 1;
io[2].iov_len = bytes;
/* Undersized write should fail */
ASSERT_EQ(-1, writev(self->data_fd, (const struct iovec *)io, 1));
ASSERT_EQ(EINVAL, errno);
/* Enable event */
self->enable_fd = open(enable_file, O_RDWR);
ASSERT_NE(-1, write(self->enable_fd, "1", sizeof("1")))
/* Full in-bounds write should work */
before = trace_bytes();
loc = DYN_LOC(0, bytes);
ASSERT_NE(-1, writev(self->data_fd, (const struct iovec *)io, 3));
after = trace_bytes();
ASSERT_GT(after, before);
/* Out of bounds write should fault (offset way out) */
loc = DYN_LOC(1024, bytes);
ASSERT_EQ(-1, writev(self->data_fd, (const struct iovec *)io, 3));
ASSERT_EQ(EFAULT, errno);
/* Out of bounds write should fault (offset 1 byte out) */
loc = DYN_LOC(1, bytes);
ASSERT_EQ(-1, writev(self->data_fd, (const struct iovec *)io, 3));
ASSERT_EQ(EFAULT, errno);
/* Out of bounds write should fault (size way out) */
loc = DYN_LOC(0, bytes + 1024);
ASSERT_EQ(-1, writev(self->data_fd, (const struct iovec *)io, 3));
ASSERT_EQ(EFAULT, errno);
/* Out of bounds write should fault (size 1 byte out) */
loc = DYN_LOC(0, bytes + 1);
ASSERT_EQ(-1, writev(self->data_fd, (const struct iovec *)io, 3));
ASSERT_EQ(EFAULT, errno);
/* Non-Null should fault */
memset(data, 'A', sizeof(data));
loc = DYN_LOC(0, bytes);
ASSERT_EQ(-1, writev(self->data_fd, (const struct iovec *)io, 3));
ASSERT_EQ(EFAULT, errno);
}
TEST_F(user, print_fmt) {
int ret;
ret = check_print_fmt("__test_event __rel_loc char[] data",
"print fmt: \"data=%s\", __get_rel_str(data)");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event __data_loc char[] data",
"print fmt: \"data=%s\", __get_str(data)");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event s64 data",
"print fmt: \"data=%lld\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event u64 data",
"print fmt: \"data=%llu\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event s32 data",
"print fmt: \"data=%d\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event u32 data",
"print fmt: \"data=%u\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event int data",
"print fmt: \"data=%d\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event unsigned int data",
"print fmt: \"data=%u\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event s16 data",
"print fmt: \"data=%d\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event u16 data",
"print fmt: \"data=%u\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event short data",
"print fmt: \"data=%d\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event unsigned short data",
"print fmt: \"data=%u\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event s8 data",
"print fmt: \"data=%d\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event u8 data",
"print fmt: \"data=%u\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event char data",
"print fmt: \"data=%d\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event unsigned char data",
"print fmt: \"data=%u\", REC->data");
ASSERT_EQ(0, ret);
ret = check_print_fmt("__test_event char[4] data",
"print fmt: \"data=%s\", REC->data");
ASSERT_EQ(0, ret);
}
int main(int argc, char **argv)
{
return test_harness_run(argc, argv);
}

168
tools/testing/selftests/user_events/perf_test.c Normal file
View File

@@ -0,0 +1,168 @@
// SPDX-License-Identifier: GPL-2.0
/*
* User Events Perf Events Test Program
*
* Copyright (c) 2021 Beau Belgrave <beaub@linux.microsoft.com>
*/
#include <errno.h>
#include <linux/user_events.h>
#include <linux/perf_event.h>
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <sys/stat.h>
#include <unistd.h>
#include <asm/unistd.h>
#include "../kselftest_harness.h"
const char *data_file = "/sys/kernel/debug/tracing/user_events_data";
const char *status_file = "/sys/kernel/debug/tracing/user_events_status";
const char *id_file = "/sys/kernel/debug/tracing/events/user_events/__test_event/id";
const char *fmt_file = "/sys/kernel/debug/tracing/events/user_events/__test_event/format";
struct event {
__u32 index;
__u32 field1;
__u32 field2;
};
static long perf_event_open(struct perf_event_attr *pe, pid_t pid,
int cpu, int group_fd, unsigned long flags)
{
return syscall(__NR_perf_event_open, pe, pid, cpu, group_fd, flags);
}
static int get_id(void)
{
FILE *fp = fopen(id_file, "r");
int ret, id = 0;
if (!fp)
return -1;
ret = fscanf(fp, "%d", &id);
fclose(fp);
if (ret != 1)
return -1;
return id;
}
static int get_offset(void)
{
FILE *fp = fopen(fmt_file, "r");
int ret, c, last = 0, offset = 0;
if (!fp)
return -1;
/* Read until empty line */
while (true) {
c = getc(fp);
if (c == EOF)
break;
if (last == '\n' && c == '\n')
break;
last = c;
}
ret = fscanf(fp, "\tfield:u32 field1;\toffset:%d;", &offset);
fclose(fp);
if (ret != 1)
return -1;
return offset;
}
FIXTURE(user) {
int status_fd;
int data_fd;
};
FIXTURE_SETUP(user) {
self->status_fd = open(status_file, O_RDONLY);
ASSERT_NE(-1, self->status_fd);
self->data_fd = open(data_file, O_RDWR);
ASSERT_NE(-1, self->data_fd);
}
FIXTURE_TEARDOWN(user) {
close(self->status_fd);
close(self->data_fd);
}
TEST_F(user, perf_write) {
struct perf_event_attr pe = {0};
struct user_reg reg = {0};
int page_size = sysconf(_SC_PAGESIZE);
char *status_page;
struct event event;
struct perf_event_mmap_page *perf_page;
int id, fd, offset;
__u32 *val;
reg.size = sizeof(reg);
reg.name_args = (__u64)"__test_event u32 field1; u32 field2";
status_page = mmap(NULL, page_size, PROT_READ, MAP_SHARED,
self->status_fd, 0);
ASSERT_NE(MAP_FAILED, status_page);
/* Register should work */
ASSERT_EQ(0, ioctl(self->data_fd, DIAG_IOCSREG, &reg));
ASSERT_EQ(0, reg.write_index);
ASSERT_NE(0, reg.status_index);
ASSERT_EQ(0, status_page[reg.status_index]);
/* Id should be there */
id = get_id();
ASSERT_NE(-1, id);
offset = get_offset();
ASSERT_NE(-1, offset);
pe.type = PERF_TYPE_TRACEPOINT;
pe.size = sizeof(pe);
pe.config = id;
pe.sample_type = PERF_SAMPLE_RAW;
pe.sample_period = 1;
pe.wakeup_events = 1;
/* Tracepoint attach should work */
fd = perf_event_open(&pe, 0, -1, -1, 0);
ASSERT_NE(-1, fd);
perf_page = mmap(NULL, page_size * 2, PROT_READ, MAP_SHARED, fd, 0);
ASSERT_NE(MAP_FAILED, perf_page);
/* Status should be updated */
ASSERT_EQ(EVENT_STATUS_PERF, status_page[reg.status_index]);
event.index = reg.write_index;
event.field1 = 0xc001;
event.field2 = 0xc01a;
/* Ensure write shows up at correct offset */
ASSERT_NE(-1, write(self->data_fd, &event, sizeof(event)));
val = (void *)(((char *)perf_page) + perf_page->data_offset);
ASSERT_EQ(PERF_RECORD_SAMPLE, *val);
/* Skip over header and size, move to offset */
val += 3;
val = (void *)((char *)val) + offset;
/* Ensure correct */
ASSERT_EQ(event.field1, *val++);
ASSERT_EQ(event.field2, *val++);
}
int main(int argc, char **argv)
{
return test_harness_run(argc, argv);
}

1
tools/testing/selftests/user_events/settings Normal file
View File

@@ -0,0 +1 @@
timeout=90

83
tools/tracing/rtla/src/osnoise.c
View File

@@ -655,6 +655,85 @@ void osnoise_put_print_stack(struct osnoise_context *context)
context->orig_print_stack = OSNOISE_OPTION_INIT_VAL;
}
/*
* osnoise_get_tracing_thresh - read and save the original "tracing_thresh"
*/
static long long
osnoise_get_tracing_thresh(struct osnoise_context *context)
{
long long tracing_thresh;
if (context->tracing_thresh != OSNOISE_OPTION_INIT_VAL)
return context->tracing_thresh;
if (context->orig_tracing_thresh != OSNOISE_OPTION_INIT_VAL)
return context->orig_tracing_thresh;
tracing_thresh = osnoise_read_ll_config("tracing_thresh");
if (tracing_thresh < 0)
goto out_err;
context->orig_tracing_thresh = tracing_thresh;
return tracing_thresh;
out_err:
return OSNOISE_OPTION_INIT_VAL;
}
/*
* osnoise_set_tracing_thresh - set "tracing_thresh"
*/
int osnoise_set_tracing_thresh(struct osnoise_context *context, long long tracing_thresh)
{
long long curr_tracing_thresh = osnoise_get_tracing_thresh(context);
int retval;
if (curr_tracing_thresh == OSNOISE_OPTION_INIT_VAL)
return -1;
retval = osnoise_write_ll_config("tracing_thresh", tracing_thresh);
if (retval < 0)
return -1;
context->tracing_thresh = tracing_thresh;
return 0;
}
/*
* osnoise_restore_tracing_thresh - restore the original "tracing_thresh"
*/
void osnoise_restore_tracing_thresh(struct osnoise_context *context)
{
int retval;
if (context->orig_tracing_thresh == OSNOISE_OPTION_INIT_VAL)
return;
if (context->orig_tracing_thresh == context->tracing_thresh)
goto out_done;
retval = osnoise_write_ll_config("tracing_thresh", context->orig_tracing_thresh);
if (retval < 0)
err_msg("Could not restore original tracing_thresh\n");
out_done:
context->tracing_thresh = OSNOISE_OPTION_INIT_VAL;
}
/*
* osnoise_put_tracing_thresh - restore original values and cleanup data
*/
void osnoise_put_tracing_thresh(struct osnoise_context *context)
{
osnoise_restore_tracing_thresh(context);
if (context->orig_tracing_thresh == OSNOISE_OPTION_INIT_VAL)
return;
context->orig_tracing_thresh = OSNOISE_OPTION_INIT_VAL;
}
/*
* enable_osnoise - enable osnoise tracer in the trace_instance
*/
@@ -716,6 +795,9 @@ struct osnoise_context *osnoise_context_alloc(void)
context->orig_print_stack = OSNOISE_OPTION_INIT_VAL;
context->print_stack = OSNOISE_OPTION_INIT_VAL;
context->orig_tracing_thresh = OSNOISE_OPTION_INIT_VAL;
context->tracing_thresh = OSNOISE_OPTION_INIT_VAL;
osnoise_get_context(context);
return context;
@@ -741,6 +823,7 @@ void osnoise_put_context(struct osnoise_context *context)
osnoise_put_stop_total_us(context);
osnoise_put_timerlat_period_us(context);
osnoise_put_print_stack(context);
osnoise_put_tracing_thresh(context);
free(context);
}

8
tools/tracing/rtla/src/osnoise.h
View File

@@ -23,6 +23,10 @@ struct osnoise_context {
long long orig_timerlat_period_us;
long long timerlat_period_us;
/* 0 as init value */
long long orig_tracing_thresh;
long long tracing_thresh;
/* -1 as init value because 0 is disabled */
long long orig_stop_us;
long long stop_us;
@@ -67,6 +71,10 @@ int osnoise_set_timerlat_period_us(struct osnoise_context *context,
long long timerlat_period_us);
void osnoise_restore_timerlat_period_us(struct osnoise_context *context);
int osnoise_set_tracing_thresh(struct osnoise_context *context,
long long tracing_thresh);
void osnoise_restore_tracing_thresh(struct osnoise_context *context);
void osnoise_restore_print_stack(struct osnoise_context *context);
int osnoise_set_print_stack(struct osnoise_context *context,
long long print_stack);

95
tools/tracing/rtla/src/osnoise_hist.c
View File

@@ -21,6 +21,7 @@ struct osnoise_hist_params {
char *trace_output;
unsigned long long runtime;
unsigned long long period;
long long threshold;
long long stop_us;
long long stop_total_us;
int sleep_time;
@@ -28,6 +29,7 @@ struct osnoise_hist_params {
int set_sched;
int output_divisor;
struct sched_attr sched_param;
struct trace_events *events;
char no_header;
char no_summary;
@@ -425,19 +427,25 @@ static void osnoise_hist_usage(char *usage)
static const char * const msg[] = {
"",
" usage: rtla osnoise hist [-h] [-D] [-d s] [-p us] [-r us] [-s us] [-S us] [-t[=file]] \\",
" [-c cpu-list] [-P priority] [-b N] [-E N] [--no-header] [--no-summary] \\",
" [--no-index] [--with-zeros]",
" usage: rtla osnoise hist [-h] [-D] [-d s] [-a us] [-p us] [-r us] [-s us] [-S us] \\",
" [-T us] [-t[=file]] [-e sys[:event]] [--filter <filter>] [--trigger <trigger>] \\",
" [-c cpu-list] [-P priority] [-b N] [-E N] [--no-header] [--no-summary] [--no-index] \\",
" [--with-zeros]",
"",
" -h/--help: print this menu",
" -a/--auto: set automatic trace mode, stopping the session if argument in us sample is hit",
" -p/--period us: osnoise period in us",
" -r/--runtime us: osnoise runtime in us",
" -s/--stop us: stop trace if a single sample is higher than the argument in us",
" -S/--stop-total us: stop trace if the total sample is higher than the argument in us",
" -T/--threshold us: the minimum delta to be considered a noise",
" -c/--cpus cpu-list: list of cpus to run osnoise threads",
" -d/--duration time[s|m|h|d]: duration of the session",
" -D/--debug: print debug info",
" -t/--trace[=file]: save the stopped trace to [file|osnoise_trace.txt]",
" -e/--event <sys:event>: enable the <sys:event> in the trace instance, multiple -e are allowed",
" --filter <filter>: enable a trace event filter to the previous -e event",
" --trigger <trigger>: enable a trace event trigger to the previous -e event",
" -b/--bucket-size N: set the histogram bucket size (default 1)",
" -E/--entries N: set the number of entries of the histogram (default 256)",
" --no-header: do not print header",
@@ -471,6 +479,7 @@ static struct osnoise_hist_params
*osnoise_hist_parse_args(int argc, char *argv[])
{
struct osnoise_hist_params *params;
struct trace_events *tevent;
int retval;
int c;
@@ -485,6 +494,7 @@ static struct osnoise_hist_params
while (1) {
static struct option long_options[] = {
{"auto", required_argument, 0, 'a'},
{"bucket-size", required_argument, 0, 'b'},
{"entries", required_argument, 0, 'E'},
{"cpus", required_argument, 0, 'c'},
@@ -497,17 +507,21 @@ static struct osnoise_hist_params
{"stop", required_argument, 0, 's'},
{"stop-total", required_argument, 0, 'S'},
{"trace", optional_argument, 0, 't'},
{"event", required_argument, 0, 'e'},
{"threshold", required_argument, 0, 'T'},
{"no-header", no_argument, 0, '0'},
{"no-summary", no_argument, 0, '1'},
{"no-index", no_argument, 0, '2'},
{"with-zeros", no_argument, 0, '3'},
{"trigger", required_argument, 0, '4'},
{"filter", required_argument, 0, '5'},
{0, 0, 0, 0}
};
/* getopt_long stores the option index here. */
int option_index = 0;
c = getopt_long(argc, argv, "c:b:d:E:Dhp:P:r:s:S:t::0123",
c = getopt_long(argc, argv, "a:c:b:d:e:E:Dhp:P:r:s:S:t::T:01234:5:",
long_options, &option_index);
/* detect the end of the options. */
@@ -515,6 +529,17 @@ static struct osnoise_hist_params
break;
switch (c) {
case 'a':
/* set sample stop to auto_thresh */
params->stop_us = get_llong_from_str(optarg);
/* set sample threshold to 1 */
params->threshold = 1;
/* set trace */
params->trace_output = "osnoise_trace.txt";
break;
case 'b':
params->bucket_size = get_llong_from_str(optarg);
if ((params->bucket_size == 0) || (params->bucket_size >= 1000000))
@@ -534,6 +559,18 @@ static struct osnoise_hist_params
if (!params->duration)
osnoise_hist_usage("Invalid -D duration\n");
break;
case 'e':
tevent = trace_event_alloc(optarg);
if (!tevent) {
err_msg("Error alloc trace event");
exit(EXIT_FAILURE);
}
if (params->events)
tevent->next = params->events;
params->events = tevent;
break;
case 'E':
params->entries = get_llong_from_str(optarg);
if ((params->entries < 10) || (params->entries > 9999999))
@@ -565,6 +602,9 @@ static struct osnoise_hist_params
case 'S':
params->stop_total_us = get_llong_from_str(optarg);
break;
case 'T':
params->threshold = get_llong_from_str(optarg);
break;
case 't':
if (optarg)
/* skip = */
@@ -584,6 +624,28 @@ static struct osnoise_hist_params
case '3': /* with zeros */
params->with_zeros = 1;
break;
case '4': /* trigger */
if (params->events) {
retval = trace_event_add_trigger(params->events, optarg);
if (retval) {
err_msg("Error adding trigger %s\n", optarg);
exit(EXIT_FAILURE);
}
} else {
osnoise_hist_usage("--trigger requires a previous -e\n");
}
break;
case '5': /* filter */
if (params->events) {
retval = trace_event_add_filter(params->events, optarg);
if (retval) {
err_msg("Error adding filter %s\n", optarg);
exit(EXIT_FAILURE);
}
} else {
osnoise_hist_usage("--filter requires a previous -e\n");
}
break;
default:
osnoise_hist_usage("Invalid option");
}
@@ -645,6 +707,14 @@ osnoise_hist_apply_config(struct osnoise_tool *tool, struct osnoise_hist_params
}
}
if (params->threshold) {
retval = osnoise_set_tracing_thresh(tool->context, params->threshold);
if (retval) {
err_msg("Failed to set tracing_thresh\n");
goto out_err;
}
}
return 0;
out_err:
@@ -751,6 +821,13 @@ int osnoise_hist_main(int argc, char *argv[])
err_msg("Failed to enable the trace instance\n");
goto out_hist;
}
if (params->events) {
retval = trace_events_enable(&record->trace, params->events);
if (retval)
goto out_hist;
}
trace_instance_start(&record->trace);
}
@@ -771,9 +848,9 @@ int osnoise_hist_main(int argc, char *argv[])
goto out_hist;
}
if (!tracefs_trace_is_on(trace->inst))
if (trace_is_off(&tool->trace, &record->trace))
break;
};
}
osnoise_read_trace_hist(tool);
@@ -781,8 +858,8 @@ int osnoise_hist_main(int argc, char *argv[])
return_value = 0;
if (!tracefs_trace_is_on(trace->inst)) {
printf("rtla timelat hit stop tracing\n");
if (trace_is_off(&tool->trace, &record->trace)) {
printf("rtla osnoise hit stop tracing\n");
if (params->trace_output) {
printf(" Saving trace to %s\n", params->trace_output);
save_trace_to_file(record->trace.inst, params->trace_output);
@@ -790,6 +867,8 @@ int osnoise_hist_main(int argc, char *argv[])
}
out_hist:
trace_events_destroy(&record->trace, params->events);
params->events = NULL;
osnoise_free_histogram(tool->data);
out_destroy:
osnoise_destroy_tool(record);

91
tools/tracing/rtla/src/osnoise_top.c
View File

@@ -23,6 +23,7 @@ struct osnoise_top_params {
char *trace_output;
unsigned long long runtime;
unsigned long long period;
long long threshold;
long long stop_us;
long long stop_total_us;
int sleep_time;
@@ -30,6 +31,7 @@ struct osnoise_top_params {
int quiet;
int set_sched;
struct sched_attr sched_param;
struct trace_events *events;
};
struct osnoise_top_cpu {
@@ -244,18 +246,24 @@ void osnoise_top_usage(char *usage)
int i;
static const char * const msg[] = {
" usage: rtla osnoise [top] [-h] [-q] [-D] [-d s] [-p us] [-r us] [-s us] [-S us] [-t[=file]] \\",
" usage: rtla osnoise [top] [-h] [-q] [-D] [-d s] [-a us] [-p us] [-r us] [-s us] [-S us] \\",
" [-T us] [-t[=file]] [-e sys[:event]] [--filter <filter>] [--trigger <trigger>] \\",
" [-c cpu-list] [-P priority]",
"",
" -h/--help: print this menu",
" -a/--auto: set automatic trace mode, stopping the session if argument in us sample is hit",
" -p/--period us: osnoise period in us",
" -r/--runtime us: osnoise runtime in us",
" -s/--stop us: stop trace if a single sample is higher than the argument in us",
" -S/--stop-total us: stop trace if the total sample is higher than the argument in us",
" -T/--threshold us: the minimum delta to be considered a noise",
" -c/--cpus cpu-list: list of cpus to run osnoise threads",
" -d/--duration time[s|m|h|d]: duration of the session",
" -D/--debug: print debug info",
" -t/--trace[=file]: save the stopped trace to [file|osnoise_trace.txt]",
" -e/--event <sys:event>: enable the <sys:event> in the trace instance, multiple -e are allowed",
" --filter <filter>: enable a trace event filter to the previous -e event",
" --trigger <trigger>: enable a trace event trigger to the previous -e event",
" -q/--quiet print only a summary at the end",
" -P/--priority o:prio|r:prio|f:prio|d:runtime:period : set scheduling parameters",
" o:prio - use SCHED_OTHER with prio",
@@ -283,6 +291,7 @@ void osnoise_top_usage(char *usage)
struct osnoise_top_params *osnoise_top_parse_args(int argc, char **argv)
{
struct osnoise_top_params *params;
struct trace_events *tevent;
int retval;
int c;
@@ -292,9 +301,11 @@ struct osnoise_top_params *osnoise_top_parse_args(int argc, char **argv)
while (1) {
static struct option long_options[] = {
{"auto", required_argument, 0, 'a'},
{"cpus", required_argument, 0, 'c'},
{"debug", no_argument, 0, 'D'},
{"duration", required_argument, 0, 'd'},
{"event", required_argument, 0, 'e'},
{"help", no_argument, 0, 'h'},
{"period", required_argument, 0, 'p'},
{"priority", required_argument, 0, 'P'},
@@ -302,14 +313,17 @@ struct osnoise_top_params *osnoise_top_parse_args(int argc, char **argv)
{"runtime", required_argument, 0, 'r'},
{"stop", required_argument, 0, 's'},
{"stop-total", required_argument, 0, 'S'},
{"threshold", required_argument, 0, 'T'},
{"trace", optional_argument, 0, 't'},
{"trigger", required_argument, 0, '0'},
{"filter", required_argument, 0, '1'},
{0, 0, 0, 0}
};
/* getopt_long stores the option index here. */
int option_index = 0;
c = getopt_long(argc, argv, "c:d:Dhp:P:qr:s:S:t::",
c = getopt_long(argc, argv, "a:c:d:De:hp:P:qr:s:S:t::T:0:1:",
long_options, &option_index);
/* Detect the end of the options. */
@@ -317,6 +331,17 @@ struct osnoise_top_params *osnoise_top_parse_args(int argc, char **argv)
break;
switch (c) {
case 'a':
/* set sample stop to auto_thresh */
params->stop_us = get_llong_from_str(optarg);
/* set sample threshold to 1 */
params->threshold = 1;
/* set trace */
params->trace_output = "osnoise_trace.txt";
break;
case 'c':
retval = parse_cpu_list(optarg, &params->monitored_cpus);
if (retval)
@@ -331,6 +356,18 @@ struct osnoise_top_params *osnoise_top_parse_args(int argc, char **argv)
if (!params->duration)
osnoise_top_usage("Invalid -D duration\n");
break;
case 'e':
tevent = trace_event_alloc(optarg);
if (!tevent) {
err_msg("Error alloc trace event");
exit(EXIT_FAILURE);
}
if (params->events)
tevent->next = params->events;
params->events = tevent;
break;
case 'h':
case '?':
osnoise_top_usage(NULL);
@@ -367,6 +404,31 @@ struct osnoise_top_params *osnoise_top_parse_args(int argc, char **argv)
else
params->trace_output = "osnoise_trace.txt";
break;
case 'T':
params->threshold = get_llong_from_str(optarg);
break;
case '0': /* trigger */
if (params->events) {
retval = trace_event_add_trigger(params->events, optarg);
if (retval) {
err_msg("Error adding trigger %s\n", optarg);
exit(EXIT_FAILURE);
}
} else {
osnoise_top_usage("--trigger requires a previous -e\n");
}
break;
case '1': /* filter */
if (params->events) {
retval = trace_event_add_filter(params->events, optarg);
if (retval) {
err_msg("Error adding filter %s\n", optarg);
exit(EXIT_FAILURE);
}
} else {
osnoise_top_usage("--filter requires a previous -e\n");
}
break;
default:
osnoise_top_usage("Invalid option");
}
@@ -425,6 +487,14 @@ osnoise_top_apply_config(struct osnoise_tool *tool, struct osnoise_top_params *p
}
}
if (params->threshold) {
retval = osnoise_set_tracing_thresh(tool->context, params->threshold);
if (retval) {
err_msg("Failed to set tracing_thresh\n");
goto out_err;
}
}
return 0;
out_err:
@@ -529,13 +599,20 @@ int osnoise_top_main(int argc, char **argv)
err_msg("Failed to enable the trace instance\n");
goto out_top;
}
if (params->events) {
retval = trace_events_enable(&record->trace, params->events);
if (retval)
goto out_top;
}
trace_instance_start(&record->trace);
}
tool->start_time = time(NULL);
osnoise_top_set_signals(params);
do {
while (!stop_tracing) {
sleep(params->sleep_time);
retval = tracefs_iterate_raw_events(trace->tep,
@@ -552,16 +629,16 @@ int osnoise_top_main(int argc, char **argv)
if (!params->quiet)
osnoise_print_stats(params, tool);
if (!tracefs_trace_is_on(trace->inst))
if (trace_is_off(&tool->trace, &record->trace))
break;
} while (!stop_tracing);
}
osnoise_print_stats(params, tool);
return_value = 0;
if (!tracefs_trace_is_on(trace->inst)) {
if (trace_is_off(&tool->trace, &record->trace)) {
printf("osnoise hit stop tracing\n");
if (params->trace_output) {
printf(" Saving trace to %s\n", params->trace_output);
@@ -570,6 +647,8 @@ int osnoise_top_main(int argc, char **argv)
}
out_top:
trace_events_destroy(&record->trace, params->events);
params->events = NULL;
osnoise_free_top(tool->data);
osnoise_destroy_tool(record);
osnoise_destroy_tool(tool);

108
tools/tracing/rtla/src/timerlat_hist.c
View File

@@ -28,7 +28,9 @@ struct timerlat_hist_params {
int output_divisor;
int duration;
int set_sched;
int dma_latency;
struct sched_attr sched_param;
struct trace_events *events;
char no_irq;
char no_thread;
@@ -428,11 +430,13 @@ static void timerlat_hist_usage(char *usage)
char *msg[] = {
"",
" usage: [rtla] timerlat hist [-h] [-q] [-d s] [-D] [-n] [-p us] [-i us] [-T us] [-s us] [-t[=file]] \\",
" [-c cpu-list] [-P priority] [-E N] [-b N] [--no-irq] [--no-thread] [--no-header] [--no-summary] \\",
" [--no-index] [--with-zeros]",
" usage: [rtla] timerlat hist [-h] [-q] [-d s] [-D] [-n] [-a us] [-p us] [-i us] [-T us] [-s us] \\",
" [-t[=file]] [-e sys[:event]] [--filter <filter>] [--trigger <trigger>] [-c cpu-list] \\",
" [-P priority] [-E N] [-b N] [--no-irq] [--no-thread] [--no-header] [--no-summary] \\",
" [--no-index] [--with-zeros] [--dma-latency us]",
"",
" -h/--help: print this menu",
" -a/--auto: set automatic trace mode, stopping the session if argument in us latency is hit",
" -p/--period us: timerlat period in us",
" -i/--irq us: stop trace if the irq latency is higher than the argument in us",
" -T/--thread us: stop trace if the thread latency is higher than the argument in us",
@@ -440,7 +444,10 @@ static void timerlat_hist_usage(char *usage)
" -c/--cpus cpus: run the tracer only on the given cpus",
" -d/--duration time[m|h|d]: duration of the session in seconds",
" -D/--debug: print debug info",
" -T/--trace[=file]: save the stopped trace to [file|timerlat_trace.txt]",
" -t/--trace[=file]: save the stopped trace to [file|timerlat_trace.txt]",
" -e/--event <sys:event>: enable the <sys:event> in the trace instance, multiple -e are allowed",
" --filter <filter>: enable a trace event filter to the previous -e event",
" --trigger <trigger>: enable a trace event trigger to the previous -e event",
" -n/--nano: display data in nanoseconds",
" -b/--bucket-size N: set the histogram bucket size (default 1)",
" -E/--entries N: set the number of entries of the histogram (default 256)",
@@ -450,6 +457,7 @@ static void timerlat_hist_usage(char *usage)
" --no-summary: do not print summary",
" --no-index: do not print index",
" --with-zeros: print zero only entries",
" --dma-latency us: set /dev/cpu_dma_latency latency <us> to reduce exit from idle latency",
" -P/--priority o:prio|r:prio|f:prio|d:runtime:period : set scheduling parameters",
" o:prio - use SCHED_OTHER with prio",
" r:prio - use SCHED_RR with prio",
@@ -477,6 +485,8 @@ static struct timerlat_hist_params
*timerlat_hist_parse_args(int argc, char *argv[])
{
struct timerlat_hist_params *params;
struct trace_events *tevent;
int auto_thresh;
int retval;
int c;
@@ -484,6 +494,9 @@ static struct timerlat_hist_params
if (!params)
exit(1);
/* disabled by default */
params->dma_latency = -1;
/* display data in microseconds */
params->output_divisor = 1000;
params->bucket_size = 1;
@@ -491,6 +504,7 @@ static struct timerlat_hist_params
while (1) {
static struct option long_options[] = {
{"auto", required_argument, 0, 'a'},
{"cpus", required_argument, 0, 'c'},
{"bucket-size", required_argument, 0, 'b'},
{"debug", no_argument, 0, 'D'},
@@ -504,19 +518,23 @@ static struct timerlat_hist_params
{"stack", required_argument, 0, 's'},
{"thread", required_argument, 0, 'T'},
{"trace", optional_argument, 0, 't'},
{"event", required_argument, 0, 'e'},
{"no-irq", no_argument, 0, '0'},
{"no-thread", no_argument, 0, '1'},
{"no-header", no_argument, 0, '2'},
{"no-summary", no_argument, 0, '3'},
{"no-index", no_argument, 0, '4'},
{"with-zeros", no_argument, 0, '5'},
{"trigger", required_argument, 0, '6'},
{"filter", required_argument, 0, '7'},
{"dma-latency", required_argument, 0, '8'},
{0, 0, 0, 0}
};
/* getopt_long stores the option index here. */
int option_index = 0;
c = getopt_long(argc, argv, "c:b:d:E:Dhi:np:P:s:t::T:012345",
c = getopt_long(argc, argv, "a:c:b:d:e:E:Dhi:np:P:s:t::T:0123456:7:8:",
long_options, &option_index);
/* detect the end of the options. */
@@ -524,6 +542,19 @@ static struct timerlat_hist_params
break;
switch (c) {
case 'a':
auto_thresh = get_llong_from_str(optarg);
/* set thread stop to auto_thresh */
params->stop_total_us = auto_thresh;
/* get stack trace */
params->print_stack = auto_thresh;
/* set trace */
params->trace_output = "timerlat_trace.txt";
break;
case 'c':
retval = parse_cpu_list(optarg, &params->monitored_cpus);
if (retval)
@@ -543,6 +574,18 @@ static struct timerlat_hist_params
if (!params->duration)
timerlat_hist_usage("Invalid -D duration\n");
break;
case 'e':
tevent = trace_event_alloc(optarg);
if (!tevent) {
err_msg("Error alloc trace event");
exit(EXIT_FAILURE);
}
if (params->events)
tevent->next = params->events;
params->events = tevent;
break;
case 'E':
params->entries = get_llong_from_str(optarg);
if ((params->entries < 10) || (params->entries > 9999999))
@@ -600,6 +643,35 @@ static struct timerlat_hist_params
case '5': /* with zeros */
params->with_zeros = 1;
break;
case '6': /* trigger */
if (params->events) {
retval = trace_event_add_trigger(params->events, optarg);
if (retval) {
err_msg("Error adding trigger %s\n", optarg);
exit(EXIT_FAILURE);
}
} else {
timerlat_hist_usage("--trigger requires a previous -e\n");
}
break;
case '7': /* filter */
if (params->events) {
retval = trace_event_add_filter(params->events, optarg);
if (retval) {
err_msg("Error adding filter %s\n", optarg);
exit(EXIT_FAILURE);
}
} else {
timerlat_hist_usage("--filter requires a previous -e\n");
}
break;
case '8':
params->dma_latency = get_llong_from_str(optarg);
if (params->dma_latency < 0 || params->dma_latency > 10000) {
err_msg("--dma-latency needs to be >= 0 and < 10000");
exit(EXIT_FAILURE);
}
break;
default:
timerlat_hist_usage("Invalid option");
}
@@ -732,6 +804,7 @@ int timerlat_hist_main(int argc, char *argv[])
struct osnoise_tool *record = NULL;
struct osnoise_tool *tool = NULL;
struct trace_instance *trace;
int dma_latency_fd = -1;
int return_value = 1;
int retval;
@@ -767,6 +840,14 @@ int timerlat_hist_main(int argc, char *argv[])
}
}
if (params->dma_latency >= 0) {
dma_latency_fd = set_cpu_dma_latency(params->dma_latency);
if (dma_latency_fd < 0) {
err_msg("Could not set /dev/cpu_dma_latency.\n");
goto out_hist;
}
}
trace_instance_start(trace);
if (params->trace_output) {
@@ -775,6 +856,13 @@ int timerlat_hist_main(int argc, char *argv[])
err_msg("Failed to enable the trace instance\n");
goto out_hist;
}
if (params->events) {
retval = trace_events_enable(&record->trace, params->events);
if (retval)
goto out_hist;
}
trace_instance_start(&record->trace);
}
@@ -795,15 +883,15 @@ int timerlat_hist_main(int argc, char *argv[])
goto out_hist;
}
if (!tracefs_trace_is_on(trace->inst))
if (trace_is_off(&tool->trace, &record->trace))
break;
};
}
timerlat_print_stats(params, tool);
return_value = 0;
if (!tracefs_trace_is_on(trace->inst)) {
if (trace_is_off(&tool->trace, &record->trace)) {
printf("rtla timelat hit stop tracing\n");
if (params->trace_output) {
printf(" Saving trace to %s\n", params->trace_output);
@@ -812,6 +900,10 @@ int timerlat_hist_main(int argc, char *argv[])
}
out_hist:
if (dma_latency_fd >= 0)
close(dma_latency_fd);
trace_events_destroy(&record->trace, params->events);
params->events = NULL;
timerlat_free_histogram(tool->data);
osnoise_destroy_tool(record);
osnoise_destroy_tool(tool);

103
tools/tracing/rtla/src/timerlat_top.c
View File

@@ -29,7 +29,9 @@ struct timerlat_top_params {
int duration;
int quiet;
int set_sched;
int dma_latency;
struct sched_attr sched_param;
struct trace_events *events;
};
struct timerlat_top_cpu {
@@ -266,10 +268,12 @@ static void timerlat_top_usage(char *usage)
static const char *const msg[] = {
"",
" usage: rtla timerlat [top] [-h] [-q] [-d s] [-D] [-n] [-p us] [-i us] [-T us] [-s us] [-t[=file]] \\",
" [-c cpu-list] [-P priority]",
" usage: rtla timerlat [top] [-h] [-q] [-a us] [-d s] [-D] [-n] [-p us] [-i us] [-T us] [-s us] \\",
" [[-t[=file]] [-e sys[:event]] [--filter <filter>] [--trigger <trigger>] [-c cpu-list] \\",
" [-P priority] [--dma-latency us]",
"",
" -h/--help: print this menu",
" -a/--auto: set automatic trace mode, stopping the session if argument in us latency is hit",
" -p/--period us: timerlat period in us",
" -i/--irq us: stop trace if the irq latency is higher than the argument in us",
" -T/--thread us: stop trace if the thread latency is higher than the argument in us",
@@ -278,8 +282,12 @@ static void timerlat_top_usage(char *usage)
" -d/--duration time[m|h|d]: duration of the session in seconds",
" -D/--debug: print debug info",
" -t/--trace[=file]: save the stopped trace to [file|timerlat_trace.txt]",
" -e/--event <sys:event>: enable the <sys:event> in the trace instance, multiple -e are allowed",
" --filter <command>: enable a trace event filter to the previous -e event",
" --trigger <command>: enable a trace event trigger to the previous -e event",
" -n/--nano: display data in nanoseconds",
" -q/--quiet print only a summary at the end",
" --dma-latency us: set /dev/cpu_dma_latency latency <us> to reduce exit from idle latency",
" -P/--priority o:prio|r:prio|f:prio|d:runtime:period : set scheduling parameters",
" o:prio - use SCHED_OTHER with prio",
" r:prio - use SCHED_RR with prio",
@@ -307,6 +315,8 @@ static struct timerlat_top_params
*timerlat_top_parse_args(int argc, char **argv)
{
struct timerlat_top_params *params;
struct trace_events *tevent;
long long auto_thresh;
int retval;
int c;
@@ -314,14 +324,19 @@ static struct timerlat_top_params
if (!params)
exit(1);
/* disabled by default */
params->dma_latency = -1;
/* display data in microseconds */
params->output_divisor = 1000;
while (1) {
static struct option long_options[] = {
{"auto", required_argument, 0, 'a'},
{"cpus", required_argument, 0, 'c'},
{"debug", no_argument, 0, 'D'},
{"duration", required_argument, 0, 'd'},
{"event", required_argument, 0, 'e'},
{"help", no_argument, 0, 'h'},
{"irq", required_argument, 0, 'i'},
{"nano", no_argument, 0, 'n'},
@@ -331,13 +346,16 @@ static struct timerlat_top_params
{"stack", required_argument, 0, 's'},
{"thread", required_argument, 0, 'T'},
{"trace", optional_argument, 0, 't'},
{"trigger", required_argument, 0, '0'},
{"filter", required_argument, 0, '1'},
{"dma-latency", required_argument, 0, '2'},
{0, 0, 0, 0}
};
/* getopt_long stores the option index here. */
int option_index = 0;
c = getopt_long(argc, argv, "c:d:Dhi:np:P:qs:t::T:",
c = getopt_long(argc, argv, "a:c:d:De:hi:np:P:qs:t::T:0:1:2:",
long_options, &option_index);
/* detect the end of the options. */
@@ -345,6 +363,19 @@ static struct timerlat_top_params
break;
switch (c) {
case 'a':
auto_thresh = get_llong_from_str(optarg);
/* set thread stop to auto_thresh */
params->stop_total_us = auto_thresh;
/* get stack trace */
params->print_stack = auto_thresh;
/* set trace */
params->trace_output = "timerlat_trace.txt";
break;
case 'c':
retval = parse_cpu_list(optarg, &params->monitored_cpus);
if (retval)
@@ -359,6 +390,17 @@ static struct timerlat_top_params
if (!params->duration)
timerlat_top_usage("Invalid -D duration\n");
break;
case 'e':
tevent = trace_event_alloc(optarg);
if (!tevent) {
err_msg("Error alloc trace event");
exit(EXIT_FAILURE);
}
if (params->events)
tevent->next = params->events;
params->events = tevent;
break;
case 'h':
case '?':
timerlat_top_usage(NULL);
@@ -396,6 +438,35 @@ static struct timerlat_top_params
else
params->trace_output = "timerlat_trace.txt";
break;
case '0': /* trigger */
if (params->events) {
retval = trace_event_add_trigger(params->events, optarg);
if (retval) {
err_msg("Error adding trigger %s\n", optarg);
exit(EXIT_FAILURE);
}
} else {
timerlat_top_usage("--trigger requires a previous -e\n");
}
break;
case '1': /* filter */
if (params->events) {
retval = trace_event_add_filter(params->events, optarg);
if (retval) {
err_msg("Error adding filter %s\n", optarg);
exit(EXIT_FAILURE);
}
} else {
timerlat_top_usage("--filter requires a previous -e\n");
}
break;
case '2': /* dma-latency */
params->dma_latency = get_llong_from_str(optarg);
if (params->dma_latency < 0 || params->dma_latency > 10000) {
err_msg("--dma-latency needs to be >= 0 and < 10000");
exit(EXIT_FAILURE);
}
break;
default:
timerlat_top_usage("Invalid option");
}
@@ -524,6 +595,7 @@ int timerlat_top_main(int argc, char *argv[])
struct osnoise_tool *record = NULL;
struct osnoise_tool *top = NULL;
struct trace_instance *trace;
int dma_latency_fd = -1;
int return_value = 1;
int retval;
@@ -559,6 +631,14 @@ int timerlat_top_main(int argc, char *argv[])
}
}
if (params->dma_latency >= 0) {
dma_latency_fd = set_cpu_dma_latency(params->dma_latency);
if (dma_latency_fd < 0) {
err_msg("Could not set /dev/cpu_dma_latency.\n");
goto out_top;
}
}
trace_instance_start(trace);
if (params->trace_output) {
@@ -567,6 +647,13 @@ int timerlat_top_main(int argc, char *argv[])
err_msg("Failed to enable the trace instance\n");
goto out_top;
}
if (params->events) {
retval = trace_events_enable(&record->trace, params->events);
if (retval)
goto out_top;
}
trace_instance_start(&record->trace);
}
@@ -590,16 +677,16 @@ int timerlat_top_main(int argc, char *argv[])
if (!params->quiet)
timerlat_print_stats(params, top);
if (!tracefs_trace_is_on(trace->inst))
if (trace_is_off(&top->trace, &record->trace))
break;
};
}
timerlat_print_stats(params, top);
return_value = 0;
if (!tracefs_trace_is_on(trace->inst)) {
if (trace_is_off(&top->trace, &record->trace)) {
printf("rtla timelat hit stop tracing\n");
if (params->trace_output) {
printf(" Saving trace to %s\n", params->trace_output);
@@ -608,6 +695,10 @@ int timerlat_top_main(int argc, char *argv[])
}
out_top:
if (dma_latency_fd >= 0)
close(dma_latency_fd);
trace_events_destroy(&record->trace, params->events);
params->events = NULL;
timerlat_free_top(top->data);
osnoise_destroy_tool(record);
osnoise_destroy_tool(top);

345
tools/tracing/rtla/src/trace.c
View File

@@ -190,3 +190,348 @@ int trace_instance_start(struct trace_instance *trace)
{
return tracefs_trace_on(trace->inst);
}
/*
* trace_events_free - free a list of trace events
*/
static void trace_events_free(struct trace_events *events)
{
struct trace_events *tevent = events;
struct trace_events *free_event;
while (tevent) {
free_event = tevent;
tevent = tevent->next;
if (free_event->filter)
free(free_event->filter);
if (free_event->trigger)
free(free_event->trigger);
free(free_event->system);
free(free_event);
}
}
/*
* trace_event_alloc - alloc and parse a single trace event
*/
struct trace_events *trace_event_alloc(const char *event_string)
{
struct trace_events *tevent;
tevent = calloc(1, sizeof(*tevent));
if (!tevent)
return NULL;
tevent->system = strdup(event_string);
if (!tevent->system) {
free(tevent);
return NULL;
}
tevent->event = strstr(tevent->system, ":");
if (tevent->event) {
*tevent->event = '\0';
tevent->event = &tevent->event[1];
}
return tevent;
}
/*
* trace_event_add_filter - record an event filter
*/
int trace_event_add_filter(struct trace_events *event, char *filter)
{
if (event->filter)
free(event->filter);
event->filter = strdup(filter);
if (!event->filter)
return 1;
return 0;
}
/*
* trace_event_add_trigger - record an event trigger action
*/
int trace_event_add_trigger(struct trace_events *event, char *trigger)
{
if (event->trigger)
free(event->trigger);
event->trigger = strdup(trigger);
if (!event->trigger)
return 1;
return 0;
}
/*
* trace_event_disable_filter - disable an event filter
*/
static void trace_event_disable_filter(struct trace_instance *instance,
struct trace_events *tevent)
{
char filter[1024];
int retval;
if (!tevent->filter)
return;
if (!tevent->filter_enabled)
return;
debug_msg("Disabling %s:%s filter %s\n", tevent->system,
tevent->event ? : "*", tevent->filter);
snprintf(filter, 1024, "!%s\n", tevent->filter);
retval = tracefs_event_file_write(instance->inst, tevent->system,
tevent->event, "filter", filter);
if (retval < 0)
err_msg("Error disabling %s:%s filter %s\n", tevent->system,
tevent->event ? : "*", tevent->filter);
}
/*
* trace_event_save_hist - save the content of an event hist
*
* If the trigger is a hist: one, save the content of the hist file.
*/
static void trace_event_save_hist(struct trace_instance *instance,
struct trace_events *tevent)
{
int retval, index, out_fd;
mode_t mode = 0644;
char path[1024];
char *hist;
if (!tevent)
return;
/* trigger enables hist */
if (!tevent->trigger)
return;
/* is this a hist: trigger? */
retval = strncmp(tevent->trigger, "hist:", strlen("hist:"));
if (retval)
return;
snprintf(path, 1024, "%s_%s_hist.txt", tevent->system, tevent->event);
printf(" Saving event %s:%s hist to %s\n", tevent->system, tevent->event, path);
out_fd = creat(path, mode);
if (out_fd < 0) {
err_msg(" Failed to create %s output file\n", path);
return;
}
hist = tracefs_event_file_read(instance->inst, tevent->system, tevent->event, "hist", 0);
if (!hist) {
err_msg(" Failed to read %s:%s hist file\n", tevent->system, tevent->event);
goto out_close;
}
index = 0;
do {
index += write(out_fd, &hist[index], strlen(hist) - index);
} while (index < strlen(hist));
free(hist);
out_close:
close(out_fd);
}
/*
* trace_event_disable_trigger - disable an event trigger
*/
static void trace_event_disable_trigger(struct trace_instance *instance,
struct trace_events *tevent)
{
char trigger[1024];
int retval;
if (!tevent->trigger)
return;
if (!tevent->trigger_enabled)
return;
debug_msg("Disabling %s:%s trigger %s\n", tevent->system,
tevent->event ? : "*", tevent->trigger);
trace_event_save_hist(instance, tevent);
snprintf(trigger, 1024, "!%s\n", tevent->trigger);
retval = tracefs_event_file_write(instance->inst, tevent->system,
tevent->event, "trigger", trigger);
if (retval < 0)
err_msg("Error disabling %s:%s trigger %s\n", tevent->system,
tevent->event ? : "*", tevent->trigger);
}
/*
* trace_events_disable - disable all trace events
*/
void trace_events_disable(struct trace_instance *instance,
struct trace_events *events)
{
struct trace_events *tevent = events;
if (!events)
return;
while (tevent) {
debug_msg("Disabling event %s:%s\n", tevent->system, tevent->event ? : "*");
if (tevent->enabled) {
trace_event_disable_filter(instance, tevent);
trace_event_disable_trigger(instance, tevent);
tracefs_event_disable(instance->inst, tevent->system, tevent->event);
}
tevent->enabled = 0;
tevent = tevent->next;
}
}
/*
* trace_event_enable_filter - enable an event filter associated with an event
*/
static int trace_event_enable_filter(struct trace_instance *instance,
struct trace_events *tevent)
{
char filter[1024];
int retval;
if (!tevent->filter)
return 0;
if (!tevent->event) {
err_msg("Filter %s applies only for single events, not for all %s:* events\n",
tevent->filter, tevent->system);
return 1;
}
snprintf(filter, 1024, "%s\n", tevent->filter);
debug_msg("Enabling %s:%s filter %s\n", tevent->system,
tevent->event ? : "*", tevent->filter);
retval = tracefs_event_file_write(instance->inst, tevent->system,
tevent->event, "filter", filter);
if (retval < 0) {
err_msg("Error enabling %s:%s filter %s\n", tevent->system,
tevent->event ? : "*", tevent->filter);
return 1;
}
tevent->filter_enabled = 1;
return 0;
}
/*
* trace_event_enable_trigger - enable an event trigger associated with an event
*/
static int trace_event_enable_trigger(struct trace_instance *instance,
struct trace_events *tevent)
{
char trigger[1024];
int retval;
if (!tevent->trigger)
return 0;
if (!tevent->event) {
err_msg("Trigger %s applies only for single events, not for all %s:* events\n",
tevent->trigger, tevent->system);
return 1;
}
snprintf(trigger, 1024, "%s\n", tevent->trigger);
debug_msg("Enabling %s:%s trigger %s\n", tevent->system,
tevent->event ? : "*", tevent->trigger);
retval = tracefs_event_file_write(instance->inst, tevent->system,
tevent->event, "trigger", trigger);
if (retval < 0) {
err_msg("Error enabling %s:%s trigger %s\n", tevent->system,
tevent->event ? : "*", tevent->trigger);
return 1;
}
tevent->trigger_enabled = 1;
return 0;
}
/*
* trace_events_enable - enable all events
*/
int trace_events_enable(struct trace_instance *instance,
struct trace_events *events)
{
struct trace_events *tevent = events;
int retval;
while (tevent) {
debug_msg("Enabling event %s:%s\n", tevent->system, tevent->event ? : "*");
retval = tracefs_event_enable(instance->inst, tevent->system, tevent->event);
if (retval < 0) {
err_msg("Error enabling event %s:%s\n", tevent->system,
tevent->event ? : "*");
return 1;
}
retval = trace_event_enable_filter(instance, tevent);
if (retval)
return 1;
retval = trace_event_enable_trigger(instance, tevent);
if (retval)
return 1;
tevent->enabled = 1;
tevent = tevent->next;
}
return 0;
}
/*
* trace_events_destroy - disable and free all trace events
*/
void trace_events_destroy(struct trace_instance *instance,
struct trace_events *events)
{
if (!events)
return;
trace_events_disable(instance, events);
trace_events_free(events);
}
int trace_is_off(struct trace_instance *tool, struct trace_instance *trace)
{
/*
* The tool instance is always present, it is the one used to collect
* data.
*/
if (!tracefs_trace_is_on(tool->inst))
return 1;
/*
* The trace instance is only enabled when -t is set. IOW, when the system
* is tracing.
*/
if (trace && !tracefs_trace_is_on(trace->inst))
return 1;
return 0;
}

23
tools/tracing/rtla/src/trace.h
View File

@@ -2,6 +2,17 @@
#include <tracefs.h>
#include <stddef.h>
struct trace_events {
struct trace_events *next;
char *system;
char *event;
char *filter;
char *trigger;
char enabled;
char filter_enabled;
char trigger_enabled;
};
struct trace_instance {
struct tracefs_instance *inst;
struct tep_handle *tep;
@@ -25,3 +36,15 @@ void destroy_instance(struct tracefs_instance *inst);
int save_trace_to_file(struct tracefs_instance *inst, const char *filename);
int collect_registered_events(struct tep_event *tep, struct tep_record *record,
int cpu, void *context);
struct trace_events *trace_event_alloc(const char *event_string);
void trace_events_disable(struct trace_instance *instance,
struct trace_events *events);
void trace_events_destroy(struct trace_instance *instance,
struct trace_events *events);
int trace_events_enable(struct trace_instance *instance,
struct trace_events *events);
int trace_event_add_filter(struct trace_events *event, char *filter);
int trace_event_add_trigger(struct trace_events *event, char *trigger);
int trace_is_off(struct trace_instance *tool, struct trace_instance *trace);

33
tools/tracing/rtla/src/utils.c
View File

@@ -10,6 +10,7 @@
#include <unistd.h>
#include <ctype.h>
#include <errno.h>
#include <fcntl.h>
#include <sched.h>
#include <stdio.h>
@@ -431,3 +432,35 @@ int parse_prio(char *arg, struct sched_attr *sched_param)
}
return 0;
}
/*
* set_cpu_dma_latency - set the /dev/cpu_dma_latecy
*
* This is used to reduce the exit from idle latency. The value
* will be reset once the file descriptor of /dev/cpu_dma_latecy
* is closed.
*
* Return: the /dev/cpu_dma_latecy file descriptor
*/
int set_cpu_dma_latency(int32_t latency)
{
int retval;
int fd;
fd = open("/dev/cpu_dma_latency", O_RDWR);
if (fd < 0) {
err_msg("Error opening /dev/cpu_dma_latency\n");
return -1;
}
retval = write(fd, &latency, 4);
if (retval < 1) {
err_msg("Error setting /dev/cpu_dma_latency\n");
close(fd);
return -1;
}
debug_msg("Set /dev/cpu_dma_latency to %d\n", latency);
return fd;
}

1
tools/tracing/rtla/src/utils.h
View File

@@ -54,3 +54,4 @@ struct sched_attr {
int parse_prio(char *arg, struct sched_attr *sched_param);
int set_comm_sched_attr(const char *comm, struct sched_attr *attr);
int set_cpu_dma_latency(int32_t latency);