We create a static array of pointers to per-CPU data. Because the cpuid
space on arm64 is not sparse there is no need to add an extra level of
indirection. Move to use mallocarray to allocate the redistributors as
a single array.
Sponsored by: Arm Ltd
The function ibv_query_device_ex is static inline, it is not exported
from the dso. With lld 16, which is much more picky about versioning and
undefined symbols, this becomes an error.
The ibv_register_driver driver symbol is explicitly versioned in
sources, it is non-existent in un-versioned object files.
Sponsored by: NVidia networking
MFC after: 1 week
This documents SIMD usage in libc for all architectures with
specific details on the new amd64 SIMD dispatch framework.
Sponsored by: The FreeBSD Foundation
Approved by: kib
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D40693
This performs very well. x86-64-v3 and x86-64-v4 kernels were written,
too, but performed worse than the baseline kernel on short strings.
These may be added at a future point in time if the performance issues
can be fixed.
os: FreeBSD
arch: amd64
cpu: 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
│ strlen_scalar.out │ strlen_baseline.out │
│ B/s │ B/s vs base │
Short 1.667Gi ± 1% 2.676Gi ± 1% +60.55% (p=0.000 n=20)
Mid 5.459Gi ± 1% 8.756Gi ± 1% +60.39% (p=0.000 n=20)
Long 15.34Gi ± 0% 52.27Gi ± 0% +240.64% (p=0.000 n=20)
geomean 5.188Gi 10.70Gi +106.24%
Sponsored by: The FreeBSD Foundation
Approved by: kib
Reviewed by: mjg jrtc27
Differential Revision: https://reviews.freebsd.org/D40693
Add a framework for selecting from one of multiple implementations
of a function based on amd64 architecture level (cf. amd64 SysV
ABI supplement).
Sponsored by: The FreeBSD Foundation
Approved by: kib
Reviewed by: jrtc27
Differential Revision: https://reviews.freebsd.org/D40693
Most em(4) devices now enjoy TSO and TSO6, matching NetBSD and Linux
defaults.
A prior commit automasks TSO on 10/100 Ethernet due to errata and other
bugs for IPv6 were fixed recently allowing this.
Mike Karels identified a performance anomaly on Intel 82574L devices.
These are multiqueue enabled on FreeBSD since the conversion to
iflib. I am investigating whether this can be fixed, in the mean time
MSI-X with checksum offloads remain default.
i219 SPT devices have an errata that downclocks the DMA engine, which
results in TSO not being able to acheive line rate. Therefore, it is
disabled on:
* Intel(R) I219-LM and I219-V SPT
* Intel(R) I219-LM and I219-V SPT-H (2)
* Intel(R) I219-LM and I219-V LBG (3)
* Intel(R) I219-LM and I219-V SPT (4)
* Intel(R) I219-LM and I219-V SPT (5)
Many lem(4) devices enjoy TSO, exceptions being 82542, 82543, 82547.
TSO6 may be possible for some chipsets but I am still working through
my testing matrix and that is hidden behind hw.em.unsupported_tso.
If you encounter issues, you may disable TSO with for example:
ifconfig em0 -tso -tso6.
I ask to be informed of any deviations from normal operation requiring
this.
Thanks to cc@ for access to emulab.net.
On a sample I219 system it saves about 16% CPU on IPv4 and 19% on IPv6.
iperf3 -Vc reported numbers:
total% user% system%
IPv4 TSO
21.3 7 14.4
21.4 6 15.4
21.5 6 15.5
IPv4 no TSO
36.8 5.4 31.4
38.5 5.1 33.5
38.2 5.7 32.6
IPv4 no TSO no TXCSUM
45.1 5.8 39.3
46 6.3 39.7
46.2 5.9 40.4
IPv6 TSO6
21.7 5.4 16.3
21.6 5.1 16.5
21.9 5.6 16.3
IPv6 no TSO6
41.2 5.2 36
41 5.1 36
40.8 5.2 35.7
IPv6 no TSO6 no TXCSUM6
49 5.9 43.1
48.8 4.9 43.9
49 5.6 43.4
Tested by: cc (lem(4)), karels (82574L)
MFC after: 3 months
Relnotes: yes
Sponsored by: BBOX.io
Differential Revision: https://reviews.freebsd.org/D41170
Hardware with more than 256 CPU cores is currently available and will
become increasingly common over FreeBSD 14's lifetime. Increase MAXCPU
in the amd64 GENERIC kernel configuration to 1024.
Earlier commits increased some related limits. These prerequisite
commits include at least:
- d7ed40243769 Increase MAX_APIC_ID safeguard to 0x800
- d1639e43c5 cpuset: increase userland maximum size to 1024
Global and allocated arrays sized by MAXCPU result in excessive bloat
on systems with lower core counts. In addition, some code used u_char
(8 bits) to hold a CPU index, which is not valid if MAXCPU is greater
than 256.
A number of recent commits addressed these sorts of issues, including
at least:
- 133935d26f pf: atomically increment state ids
- 74ac712f72 vmm: Dynamically allocate a couple of per-CPU state save areas
- 78cfa762eb callout: Move per-CPU callout state into the dpcpu region
- 42f722e721 amd64: store pcids pmap data in pcpu zone
- 9801e7c275 smp_topo: dynamically allocate group array
- 9fb6718d1b smp: Dynamically allocate the stoppcbs array
- 2bb16c6352 x86: retire use of intr_bind
There are some additional allocations still to be converted and
more scalability work is required to make effective use of very high
core count systems, but this change allows us to boot on these systems
and provides a Kernel Binary Interface (KBI) for the FreeBSD 14 release
that supports these configurations.
Special thanks to AMD for providing hardware to test these changes.
PR: 269572
Reviewed by: des
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36838
This code was used by the first incarnation of wg(4) and is dead ever
since f187d6dfbf has removed the latter
again. Moreover, this code matched iflib(4) like a square peg fits in
a round hole, was incomplete and despite some hacks still tailored to
VPC and wg(4) but not generic. In effect, this reverts the following:
09f6ff4f1a (w/ its "ancillary changes")
9aeca213241f93e931d90f9544d03e0dd691b412
Reviewed by: erj, kbowling
Differential Revision: <https://reviews.freebsd.org/D41196>
`intr_bind(u_int vector, u_char cpu);` looked suspicious since
everywhere else "cpu" is a u_int and >256 processors isn't unreasonable
now. `intr_bind()` is not used anywhere in FreeBSD (now, after commit
bf42f3738087). Time to remove.
Relnotes: Yes
Reviewed by: mjg
Differential Revision: https://reviews.freebsd.org/D36901
The vlan setting is independent for each interface. Use VFTA table in
'struct ixgbe_softc' that is already defined.
This pull request fixes following bug scenario.
create ixv0.10
create ixv1.10
destroy ixv1.10
create ixv0.11
ixv0.10 no longer receives vlan 10 packets.
In this case, destroying ixv1.10 affects to ixv0.
MFC after: 1 week
Pull Request: https://github.com/freebsd/freebsd-src/pull/774
This removes the __ieee754_ prefix from a number of the math functions.
msun/src/math_private.h contains the statement that
/*
* ieee style elementary functions
*
* We rename functions here to improve other sources' diffability
* against fdlibm.
*/
#define __ieee754_sqrt sqrt
...
Here, fdlibm refers to https://netlib.org/fdlibm. It is seen from
https://netlib.org/fdlibm/readme that this prefix was used to
differentiate between different standards:
Wrapper functions will twist the result of the ieee754
function to comply to the standard specified by the value
of _LIB_VERSION
if _LIB_VERSION = _IEEE_, return the ieee754 result;
if _LIB_VERSION = _SVID_, return SVID result;
if _LIB_VERSION = _XOPEN_, return XOPEN result;
if _LIB_VERSION = _POSIX_, return POSIX/ANSI result.
(These are macros, see fdlibm.h for their definition.)
AFAICT, FreeBSD has never supported these wrappers. In addition, as C99,
principally the long double, functions were added to libm, this
convention was not maintained. Given that only 148 of 324 files under
lib/msun contain a "Copyright (C) 1993 by Sun Microsystems" statement,
the removal of the __ieee754_ prefix provides consistency across all
source files.
The last time someone compared lib/msun to fdlibm appears to be
commit 3f70824172
Author: David Schultz <das@FreeBSD.org>
Date: Fri Feb 4 18:26:06 2005 +0000
Reduce diffs against vendor source (Sun fdlibm 5.3).
The most recent fdlibm RCS string that appears in a Sun Microsystem
copyrighted file is date "95/01/18". With Oracle Corporation's
acquisition of Sun Microsystems in 2009, it is unlikely that fdlibm will
ever be updated. A search for fdlibm at https://opensource.oracle.com/
yields no hits.
Finally, OpenBSD removed the use of this prefix over 21 years ago. pSee
revision 1.6 of OpenBSD's math_private.h.
Note: this does not drop the __ieee754_ prefix from the trigonometric
argument reduction functions, e.g., __ieee754_rem_pio2. These functions
are internal to the libm and exported through Symbol.map; and thus,
reserved for the implementation.
PR: 272783
MFC after: 1 week
When spibus is attached as child of Intel SPI controller it scans all
ACPI nodes for "SPI Serial Bus Connection Resource Descriptor" described
in section 19.6.126 of ACPI specs.
If such a descriptor is found, SPI child is added to spibus, it's SPI
chip select, mode, clock, IRQ resource and ACPI handle are added to ivars.
Existing ACPI bus-hosted child is deleted afterwards.
Apple ACPI SPI extensions are supported.
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D41248
Required for Apple and Microsoft -compatible HID-over-SPI drivers.
Most logic was already implemented in commit 3c08673438
"spibus: extend API: add cs_delay ivar, KEEP_CS and NO_SLEEP flags".
It dissallowed driver sleeps in the interrupt context. This commit
extends this feature to handle ddb/kdb context with following:
- Skip driver locking if SPI functions were called from kdb/ddb.
- Reinitialize controller if kdb/ddb initiated SPI transfer has
interrupted another already running one. Does not work very
reliable yet.
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D41247
Some devices like Apple HID-over-SPI may contain more than one report
descriptors necessitating creation of multiple hidbus children.
Add indentificator of child devices to distinct them.
No functional changes intended.
Differential Revision: https://reviews.freebsd.org/D41246
IRQ Resource is allocated on spibus(4). We must release it here too
rather than propagate request down the tree.
Fixes: 4dd8db62e9 ("Add IRQ resource to SPIBUS")
MFC after: 1 week
Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D41243
By letting vm_phys_enqueue_contig handle the case when npages == 0,
the callers can stop checking it, and the compiler can stop
zero-checking with every call to ffs(). Letting vm_phys_enqueue_contig
call vm_phys_enqueue_contig for part of its work also saves a few
bytes.
The amd64 object code shrinks by 128 bytes.
Reviewed by: kib (previous version)
Tested by: pho
Differential Revision: https://reviews.freebsd.org/D41154
Free allocated per-CPU structures in the pcpu_fini and finalize methods.
While here, add debug trace entries to these methods.
Reviewed by: jkoshy, andrew
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41269
Free allocated per-CPU structures in the pcpu_fini and finalize methods.
While here, add debug trace entries to these methods.
Reviewed by: jkoshy, andrew
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41268
Add web links to intro(9) and the Architecture Handbook.
Reviewed by: lwhsu, emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41253
This page has existed as a placeholder since its creation in 1995. It
does not provide a useful introduction to the content in this section.
Reimagine it as a top-level overview page containing brief descriptions
and links to existing pages in section 9. It is roughly organized into
sub-sections, grouped by topic or subsystem. In other words, the page is
meant to function as a map to other content.
There is a balance to be found here between providing as many links as
possible and keeping the page concise and searchable. In general the aim
is to reference pages which provide the best entry point to a particular
topic. For example, a link is given to locking(9), but not to the
specific lock pages such as mutex(9) or rwlock(9).
NetBSD has done something similar with their intro(9), so some
inspiration has been taken from there, although their content doesn't
align that closely with what we have.
I have done a thorough review of our existing pages and formed these
subsections around them, but they are meant to evolve.
PR: 270481
Reviewed by: imp, emaste
MFC after: 3 weeks
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41104
This page documents 'struct bio', so this can be helpful with finding or
referencing it.
Reviewed by: imp, markj
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D41252
The driver belabors the point about unsupported SFPs, printing multiple
times on link up. Limit it to once.
Reviewed by: markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D39785
This feature masks TSO capability when a link comes up at 10 or 100mbit
due to errata on the chips. This behavior matches previous versions of
FreeBSD as well as NetBSD and Linux.
A tunable, hw.em.unsupported_tso may be set if the admin desires to
disabling automasking and configure TSO settings manually.
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D41170
patch to fix half-cycle trigonometric functions
Paul Zimmermann, a MPFR developer, contacted me about large errors in
the half-cycle trigonometric functions. I've have investigated these
issues and developed the attached patch. The float, double, and ld80
(long double) changes have been tested.
Caveat emptor: The ld128 changes have not been compiled. The ld128
changes have not been tested. I do not have access to a system that
uses ld128 floating point.
Here is an itemized list of changes:
* lib/msun/src/math_private.h:
. Add fast floor macros to compute the integer part of |x| for
0 <= |x| 01xp(N-1), where N is the precision of the type of x.
These macros are used in the half-cycle trigonometric functions
(e.g., sinpi(x)).
. The FFLOOR80 macros is used with the Intel 80-bit extended double
functions. This macors corrects an off-by-one error, which led to
enormous error for |x| > 0x1p32.
* lib/msun/src/s_cospif.c:
* lib/msun/src/s_cospi.c:
* lib/msun/ld80/s_cospil.c:
. Update Copyright years.
. Use FFLOOR*() macro to get integer part of |x|.
. Correct handle the range 0x1p(N-1) <= |x| < 0x1pN. Here, one needs
to determine if the integral value of |x| is even or odd to choose
+1 or -1. If |x| >= 0x1pN, always return +1.
* lib/msun/src/s_sinpif.c:
* lib/msun/src/s_sinpi.c:
* lib/msun/ld80/s_sinpil.c:
. Update Copyright years.
. Use FFLOOR*() macro to get integer part of |x|.
* lib/msun/src/s_tanpif.c:
* lib/msun/src/s_tanpi.c:
* lib/msun/ld80/s_tanpil.c:
. Update Copyright years.
. For +-0.5, return +-inf. Previously, tanpi[fl]() returned an NaN.
. Use FFLOOR*() to get integer part of |x|. Need to determine if the
integer part is even or odd. This is used to set +-0 for |x|
integral
and +-inf for (n+1/2).
. For 0x1p(N-1) <= |x| < 0x1pN need to determine if x is an even or
odd
integer to select +0 or -0. For |x| >= 0x1pN, it is always an even
integer, select 0.
. Note, tanpi[fl](x) is an odd function, so one needs to consider
tanpi[fl](-|x|) = - tanpi[fl](|x|).
* lib/msun/ld128/s_cospil.c:
* lib/msun/ld128/s_sinpil.c:
* lib/msun/ld128/s_tanpil.c:
. Update Copyright years.
. These routines use an FFLOOR128 macros, which likely should be
replaced by a bit twiddling algorithm.
. The same considerations above are applied to 0x1p112 <= |x| <
0x1p113,
and |x| >= 0x1p113 cases.
. Note, even and odd determination used fmodl(x,2.), which is likely
slow.
PR: 272742
MFC after: 1 week
Yuri Pankov