Consistently provide ffs/fls using builtins

Use of compiler builtin ffs/ctz functions will result in optimized
instruction sequences when possible, and fall back to calling a function
provided by the compiler run-time library. We have slowly shifted our
platforms to take advantage of these builtins in 60645781d6 (arm64),
1c76d3a9fb (arm), 9e319462a0 (powerpc, partial).

Some platforms still rely on the libkern implementations of these
functions provided by libkern, namely riscv, powerpc (ffs*, flsll), and
i386 (ffsll and flsll). These routines are slow, as they perform a
linear search for the bit in question. Even on platforms lacking
dedicated bit-search instructions, such as riscv, the compiler library
will provide better-optimized routines, e.g. by using binary search.

Consolidate all definitions of these functions (whether currently using
builtins or not) to libkern.h. This should result in equivalent or
better performing routines in all cases.

One wart in all of this is the existing HAVE_INLINE_F*** macros, which
we use in a few places to conditionally avoid the slow libkern routines.
These aren't easily removed in one commit. For now, provide these
defines unconditionally, but marked for removal after subsequent
cleanup.

Removal of the now unused libkern routines will follow in the next
commit.

Reviewed by:	dougm, imp (previous version)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D40698
This commit is contained in:
Mitchell Horne
2023-07-06 14:45:39 -03:00
parent e8404a72c5
commit a89262079e
9 changed files with 57 additions and 238 deletions
-57
View File
@@ -183,63 +183,6 @@ void cpu_reset (void) __attribute__((__noreturn__));
extern int arm_dcache_align;
extern int arm_dcache_align_mask;
#define HAVE_INLINE_FFS
static __inline __pure2 int
ffs(int mask)
{
return (__builtin_ffs(mask));
}
#define HAVE_INLINE_FFSL
static __inline __pure2 int
ffsl(long mask)
{
return (__builtin_ffsl(mask));
}
#define HAVE_INLINE_FFSLL
static __inline __pure2 int
ffsll(long long mask)
{
return (__builtin_ffsll(mask));
}
#define HAVE_INLINE_FLS
static __inline __pure2 int
fls(int mask)
{
return (mask == 0 ? 0 :
8 * sizeof(mask) - __builtin_clz((u_int)mask));
}
#define HAVE_INLINE_FLSL
static __inline __pure2 int
flsl(long mask)
{
return (mask == 0 ? 0 :
8 * sizeof(mask) - __builtin_clzl((u_long)mask));
}
#define HAVE_INLINE_FLSLL
static __inline __pure2 int
flsll(long long mask)
{
return (mask == 0 ? 0 :
8 * sizeof(mask) - __builtin_clzll((unsigned long long)mask));
}
#else /* !_KERNEL */
static __inline void