Modulates potassium channel activity via direct interaction interaction with a potassium channel (binding or modification).