Yubikeys now can properly handle ssh keys, the old way via gpg-agent is not needed any more.

tl;dr:
a.) run setup-yubikey, giving a PIN and touching yubikey button a few times. It also tells you how to setup client/server
b.) run ssh-add -K before using ssh

Details:

You need new ssh keys to do that. Either ecdsa-sk or ed25519-sk (later one preferred, it’s newer, thought to be more secure, however slightly less compatible). The -sk indicates those are yubikey keys. Note you need openssh 8.2 or newer.

FIDO2 tends to require the user presence when logging in (i.e. pressing the yubikey button).
You can disable that, but it has to be disabled on both ssh key and server. This is obviously a fuckup: If you disable it on the key alone, all servers in standard configuration will block login. If you disable it on the server alone, it has no effect, as the key will still require it.

Optional:
You can store the private key completely on the Yubikey, this is called resident (or discoverable) keys. In order to do so, you must set a pin to the key. Otherwise the Yubikey would allow login without any further credential, which is obviously a bad idea. If you want to store more than one key (see no touch solution below), you must set the user when generating the key.

Solution:
We generate two ssh keys.
A.) not requiring touch, as default yubikey ssh key
B.) requiring touch, in a seperate keyfile and a seperate slot on the yubikey
Then we add only one of those two to servers authorized_keys, in case of key A.) we preprend a “no-touch-required”, in case of B.) we set the alterantive key to use in client sshs ~/.ssh/config.d/SERVERNAME

setup-yubikey script

#!/bin/bash
set -eu

if [[ -f ~/.ssh/id_ed25519_sk ]] ; then
   echo "yubikey ssh keys already setup, skipping"
else  
   if ykman fido info | grep -q "PIN is set" ; then
       echo "yubikey is already PIN protected. Good!"
   else
       echo "Setting initial pin to yubikey"
       ykman fido access change-pin
   fi
   echo ""
   echo "generating default no touch key"
   KEYUSER=`whoami`+no-touch@`hostname`
   ssh-keygen -t ed25519-sk -O resident -O no-touch-required -C $KEYUSER -O user=$KEYUSER -f ~/.ssh/id_ed25519_sk
   echo ""
   echo "generating touch key"
   KEYUSER=`whoami`+touch@`hostname`
   ssh-keygen -t ed25519-sk -O resident -C $KEYUSER -O user=$KEYUSER -f ~/.ssh/id_ed25519_sk_touch
   echo ""
   echo "Add to ~/.ssh/authorized_keys2 to enable login without touch of button:"
   echo -n "no-touch-required " ; cat ~/.ssh/id_ed25519_sk.pub
   echo  ""
   echo "Add to ~/.ssh/authorized_keys2 to enable login with touch of button:"
   cat ~/.ssh/id_ed25519_sk_touch.pub
   echo "and then add something to client ~/.ssh/config.d/SERVERNAME"
   echo "Host SERVERNAME"
   echo "    IdentityFile ~/.ssh/id_ed25519_sk_touch"
fi