The fields in SCTLR_EL1 and HCR_EL2 for enabling MTE are set, and if the
ID_AA64PFR1_EL1 register shows MTE is present, the GCR_EL1 register is
also configured, and the two TFSR registers which hold pending tag check
faults are cleared.
Reviewed by: andrew
Sponsored by: Arm Ltd
Signed-off-by: Harry Moulton <harry.moulton@arm.com>
Differential Revision: https://reviews.freebsd.org/D55946
Add system register and bit field definitions for Memory Tagging
Extension (MTE) in ARMv8.5.
Reviewed by: andrew
Sponsored by: Arm Ltd
Signed-off-by: Harry Moulton <harry.moulton@arm.com>
Co-authored-by: Andrew Turner <andrew@FreeBSD.org>
Differential Revision: https://reviews.freebsd.org/D55945
Cleanup the definitions in armreg.h for the CSSIDR_EL1, CLIDR_EL1 and
CSSELR_EL1 system register to prepare for additional bitfeilds for
Memory Tagging Extension (MTE).
Reviewed by: andrew
Sponsored by: Arm Ltd
Signed-off-by: Harry Moulton <harry.moulton@arm.com>
Differential Revision: https://reviews.freebsd.org/D55944
Support changing the property of a DMAP page that holds it's own page
table entry.
Because we need to perform a break-before-make sequence to change the
properties of pages a page that also holds it's own page table entry
will fault in the make part of the sequence.
Handle this by mapping the page with a temporary mapping as we already
do when demoting a superpage.
Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D55943
When modifying mappings in pmap we may need to perform a
break-before-make sequence. This creates an invalid mapping, then
recreates it with the changes.
When modifying DMAP mappings we may be changing the mapping that
contains its own page table then after breaking the old entry we are
unable to create the new entry.
To fix this create a map that can be used & won't be affected by the
break-before-make sequence.
Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D56306
* migrate the MSM8916 (snapdragon 410) support to qcom_gcc
* add the full qcom_gcc / qcom_clk list to files.arm64, replacing
the MSM8916 stub in sys/arm64/qualcomm .
Differential Revision: https://reviews.freebsd.org/D49706
sizeof(*sc->sc_ipimasks) * mp_maxid + 1 is parsed as
(sizeof(*sc->sc_ipimasks) * mp_maxid) + 1, so the buffers were one byte
short of a full (mp_maxid + 1) element count. Multiply by (mp_maxid + 1)
for sc_ipimasks and sc_cpuids.
Signed-off-by: Weixie Cui <cuiweixie@gmail.com>
Reviewed-by: kevans, ngie
Pull-Request: https://github.com/freebsd/freebsd-src/pull/2112
The kernel_pmap lock is a bit special: it does not need the DUPOK flag,
and it really belongs to a different lock class. If it belongs to the
same class as regular pmap locks, then witness may report warnings when
performing UMA allocations under a regular pmap lock, if the allocation
triggers a pmap_growkernel() call.
Replace instances of PMAP_LOCK_INIT(kernel_pmap) with inline mtx_init()
calls to silence some witness warnings for harmless behaviour I see with
some uncommitted test programs.
Reviewed by: alc, kib
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D56185
The ROUTE_MPATH compile option was introduced to
test the new multipath implementation.
Since compiling it has no overhead and it's enabled
by default, remove it.
Reviewed by: melifaro, markj
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D55884
ZSTDIO is not just for core dumps, and it is now required by ZFS, so
move it (and GZIO, which is still just for dumps) to a new “Compression
support” section in GENERIC, and add them both to MINIMAL.
MFC after: 1 week
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D55940
Rather than each exception calling userret use a common copy. As
syscallret already calls userret we need to skip it in that case.
Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D55250
It has been reported that the overhead of repeating all TLBI
instructions is too large [1]. The Software Developer Errata Notices
(SDEN) for the relevant Arm CPUs have been updated so a single
"tlbi vale1is, xzr" followed by "dsb ish" is sufficient to work around
the issues.
Replace the places we repeat TLBI instructions with the new sequence.
[1] https://lore.kernel.org/linux-arm-kernel/20260218164348.2022831-1-mark.rutland@arm.com/
Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D55646
When Top Byte Ignore (TBI) is enabled in a processor we need to pass
a canonical address to the vm subsystem.
Reviewed by: alc, kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D55619
The only new register is read-only. As the kernel just passes the
registers to the guest directly no further change should be needed.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D51764
The PMUVer field of ID_AA64DFR0 contains an unsigned version of the
Performance Monitors Extension, but it is currently treated as signed.
Change it to unsigned.
Reviewed by: andrew
Sponsored by: Arm Ltd
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
Pull Request: https://github.com/freebsd/freebsd-src/pull/2062
The function pmap_is_valid_memattr(pmap, mode) checks whether the
given variable mode is between the two constant values
VM_MEMATTR_DEVICE and VM_MEMATTR_WRITE_THROUGH.
After the code for this function was written, the value of
VM_MEMATTR_DEVICE changed from 0 to 4. Since VM_MEMATTR_WRITE_THROUGH
is still 3, the condition is always false.
This patch changes the condition to check whether mode is equal to any
of the VM_MEMATTR* constants.
Reviewed by: andrew, tuexen
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D55534
Fix spelling of NXP, which is not NPX, in two places.
Improves: a4e30909ec
Reported by: Kevin Bowling (kevin.bowling kev009.com), qoriq_dw_pci.c
MFC after: 3 days
The product series is called NXP QorIQ Layerscape. Remove the extra 'e.
MFC after: 3 days
Reviewed by: mmel, emaste
Differential Revision: https://reviews.freebsd.org/D55388
This changes the in-kernel ABI, but that is permitted across major
versions and is cleaner than leaving these around forever.
Differential Revision: https://reviews.freebsd.org/D54160
Some IVAR indices are special in that they have global meaning across
multiple buses where as other IVARs are always private to the local bus.
Try to document this a bit and add constants for the various ranges to
avoid future conflicts.
This is a no-op, but IVAR indices are now generally defined as enums
as that makes it easier to define them in terms of ranges.
Reviewed by: imp, royger, andrew
Differential Revision: https://reviews.freebsd.org/D54159
Reimplement pagezero() as ifunc. As pagezero() is only used in pmap.c,
move ifunc to that file.
Reviewed by: andrew
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D54946
Enable the use of MOPS implementations of memset, memcpy and memmove within
the kernel. Fix pre-ifunc resolution uses of these functions.
Reported by: andrew
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D55051
It now panics when it's unable to find the register, so doesn't need
to return a status.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D55106
It now only returns true so this can be assumed and doesn't need to be
checked.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D55105
This will allow for callers to be simplified & not need to check the
return status.
Keep the return type for now so this can be MFCd without breaking the
KBI.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D55104
These are needed when enabling CPU features so should be processed
earlier in the boot.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D55103
This adds padconf files for the two gpio ranges on the H616 SOC and adds
the SOC include.
Reviewed by: manu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D54915
There are programs that depend on this structure (e.g. kernel debuggers)
that breaks when the ABI changes.
Signed-off-by: Minsoo Choo <minsoochoo0122@proton.me>
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D55149
This changes STACKALIGN to be type-preserving when operating on
pointers.
Reviewed by: brooks, kib
Effort: CHERI upstreaming
Sponsored by: AFRL, DARPA
Differential Revision: https://reviews.freebsd.org/D54920
If there is no ITS device so no memory resources the gicv3 driver would
exit attaching early. This caused the vgic driver to also not be
attached, even if it could.
Reported by: novel@
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D54791
Set the Fine-grained trap registers to trap any features we don't
support. These are expected to be more useful when we support nested
virtualisation, so for now just the base features and GICv3 are not
trapped.
As nested virtualisation will require VHE we only set the fine-grained
trap registers when VHE is used.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D54687
Add the macros and detection for Fine-grained traps (FEAT_FGT and
FEAT_FGT2).
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D54686
The coredump logic calls get_arm64_sve twice: once to get the note size,
and once to get the data. The note size calculation depended on the
volatile `PCB_FP_SVEVALID` flag. If this flag was cleared between the
two calls (e.g., due to a context switch clearing the flag to comply
with the ABI), the second call would expect a smaller buffer size than
the first, triggering a KASSERT panic ("invalid size").
Fix this by ensuring the SVE state is saved to the PCB before we decide
whether to use SVE or VFP.
PR: 292195
Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D54532
It is nop for 4BSD.
Reviewed by: olce
Tested by: pho
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D54831
This function handles invalid intermediate PTP entries, but there's no
reason for this: the passed range should be mapped. In particular, we
assert that all L3 entries encountered are mapped. So let's simplify
the code a bit. No functional change intended.
Reviewed by: andrew
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D54738
We should free the allocated ASID if smmu_init_cd() fails.
Move the allocation of "domain" to simplify the first error path.
Reported by: Kevin Day <kevin@your.org>
Reviewed by: br
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D54676
On read, these registers' fields return 1 if forwarding of the
corresponding interrupt is enabled, and 0 otherwise. The test in
read_enabler() was inverted.
Reported by: Kevin Day <kevin@your.org>
Reviewed by: andrew
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D54678
Detect presence of FEAT_MOPS, and enable instruction set and set HWCAP2 flag
if present.
Add handler for MOE exceptions.
Reviewed by: andrew
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D54558
Harry Moulton