By the way, we use a security fob at work for that. Seems to work fairly well. The private key never leaves the fob, you have to press a button to sign anything, and every once in a while you have to enter your passphrase.
passphrase is set/unset locally and not communicated to the server if it is present/used/etc. Could be disabled. along with various other things.
HW token with ssh key inside is probably the best. The annoying thing is devices w/o USB. For iOS devices and android devices which support it it's probably better to just use the HW sec features. Something which did bt 4.0le and maybe had a single local LED and button would be better still.
The newer small ones work really well. You just leave them permanently in a USB port. Every once in a while you have to enter your passphrase to keep them activated (eg reboot), normally you only need to touch them to sign / log-in. The requirement for touching comes from the fob itself, and can't be overridden by our computer.
For most operations you only need your gnubby. For some more sensitive ones, we require password + gnubby touch. (You are allowed to reuse your password as the gnubby activation password.)
At first I thought that leaving the gnubby permanently in the PC would weaken security, but essentially it just means that your PC (including gnubby) is your second factor.
You can have more than one gnubby. We recommend one per computer you are using. We allow falling back to the Google Authenticator app on your phone. (It's less convenient, and potentially phishable, but otherwise secure enough.)
The earlier fobs were technically usb keyboards and were just outputting a six digit string when touched (equivalent in security to the app). The new fobs do a little cryptographic dance with the website, and are thus more secure.
From my user's point of view, it's working very well. It saves me typing my password every two minutes. And the security guys assure me it's more secure, too.
Wouldn't a passphrase be a second factor?
By the way, we use a security fob at work for that. Seems to work fairly well. The private key never leaves the fob, you have to press a button to sign anything, and every once in a while you have to enter your passphrase.