APM Cookbook – Okta MFA Integration

2016-09-12_21-41-09Since the launch of the Okta and F5 Integration Guide I’ve seen interest in leveraging this partnership take off.  One aspect I’ve enjoyed is watching how customers address pain points they were not able to address previously.  For example, providing multi-factor authentication (MFA) for Microsoft Exchange Outlook Web Access (OWA).

This particular customer standardized on Okta’s MFA solution but
OWA was behind Microsoft Threat Management Gateway (TMG) and could not easily integrate with Okta.  For this solution F5’s Access Policy Manager (APM) will replace the TMG servers  and leverage Okta’s on-premises RADIUS agent for MFA via Okta Verify, which supports push notification – by far my favorite feature.

I’ve included a video below that walks through the process of configuring Okta for RADIUS based multifactor as well as configuring APM to leverage Okta’s RADIUS agent.

Okta Configuration

On the Okta administrator portal you’ll need to create a new Okta Sign-on policy: Security -> Policies.  Once you name the new policy you’ll need to add a rule:


The crucial part here is to select RADIUS for the And Authenticates via option.

F5 Configuration

The F5 APM configuration is pretty straight forward since you can use the built-in VPE macro template for RADIUS authentication but we’ll need to create a RADIUS AAA object first.


Once the RADIUS AAA object is created go ahead and create a new Access Profile and customize your VPE as shown below – for detailed steps please watch the attached video.


Pretty easy solution and we’re just scratching the surface on what is possible.  Can’t wait to start playing with Okta’s API via iRules LX!

Kerberos is Easy – Part 2

kids coding

Yes my friends, this post has been long overdue. Life, work and all the other good excuses got in the way. However, there is nothing like a friend calling you out with a “WTF I need part 2” to get the motivation and kerberos mana flowing again. So where did we leave off? In part 1 we discussed some of the most common issues with Kerberos authentication and the necessity to break the problem down to client-side vs server-side authentication.

In part 2 we will look at some of the first troubleshooting steps I take to determine why nothing is working – yes, this happens to me too.

ADTest is Your New BF4L

You know those people that open a web browser to see if their Internet works… yea, don’t be that person. Open a terminal, check if you can ping your gateway, public DNS server, etc. and then, only then, open a browser.

ADTest is your equivalent of ping. Don’t assume because you configured an Active Directory AAA object that authentication is just going to work. Please open a console and verify that Kerberos authentication against the AD server is working with ADTest. Check out my APM Troubleshooting with ADTest for more information.

Time is not on Your Side

If you have worked with Kerberos before you know it is supper picky about time drift. If ADTest just won’t work and you can’t figure out why ensure the BIG-IP’s time matches the KDC. If you need to adjust the BIG-IP follow the F5 SOL3381.

These Are Not the KDCs You’re Looking For

Ever been in a multi-domain environment and the AD admin swears the KDC you’re talking to is the correct one; never mind the “Kerberos Principal Unknown” error you keep getting. So if ADTest doesn’t work then we need to ensure the AD server we’re talking to is a KDC for the expected realm. Now, if someone can RDP into the server this can be ruled out pretty quickly, but when in life is anything that easy.

So it’s helpful to use nslookup to find all KDCs for the intended domain and ensure the IP you were give is in this list – example below:

C:\Users\user>nslookup -type=SRV _kerberos._tcp.dc._msdcs.f5demo.com

Server: ad1.f5demo.com


_kerberos._tcp.dc._msdcs.f5demo.com SRV service location:

priority = 0

weight = 100

port = 88

svr hostname = ad1.f5demo.com

ad1.f5demo.com internet address =

ad1.f5demo.com internet address =

Playing Go Fish with SPNs

You have to admire how Microsoft took something as complicated as Kerberos and made it trivial to deploy and manage inside of Active Directory. Having managed MIT and MS version of Kerberos myself I felt a little guilty after setting up my first AD server without hours of troubleshooting issues. However, with that ease of use Microsoft also made it easy to shoot your own foot off if you have no basic understanding of Kerberos.

Think of the KDC as a key, value pair database. The KDC will let you store multiple keys of the same value even though you shouldn’t do that. So if you have multiple SPN entries in AD you are not guarantied that a request for a ticket will return the value you’re looking for. This typically presents itself in APM as authentication works onetime and not the other.

An easy way to check this is to log into a domain machine and issue:

setspn -X

This will print out any duplicate SPNs in your KDC. If the SPN you are working with appears in this list then you need to correct this issue. The easiest way I find is to delete the service account you have created for APM and just use the service account the web server application pool is using.

Cached Tickets

APM caches Kerberos tickets for both client side Kerberos authentication and server side Kerberos SSO. If you’re troubleshooting Kerberos be sure to clear these caches after you’ve made modifications.

For Kerberos AAA:

bigstart restart apd rba

For Kerberos SSO:

bigstart restart websso

I’ll Just Do It Myself

If you are working with Kerberos SSO then you have to sometimes determine if the issue is with APM obtaining the token or the web server not accepting the token. If you have your WebSSO logging set to debug and Kerberos SSO is working then you should see


If not, then there are a few CLI commands you can use to simulate the request APM makes to the KDC.

First, remove all kerberos tickets.


Second, obtain a kerberos ticket as the AD delegation account (if this works you won’t get a response)

kinit -f <SPN of AD delegation account>

Finally, test if the ticket you obtain has delegation capabilities

knvo -C -U <username> <SPN of AD delegation account>

If you receive a key version number on the kvno command then everything is working and it proves that ASREQ and RSREQ work. So the issue more than likely is on the web server side and not the F5 – more on this in part 3.

F5 APM and Okta Integration

I’m happy to announce the F5 APM and Okta integration guide has been published on Okta’s website.  I’ve been playing with this solution for the past 4 months and I have to say it’s pretty cool.  F5 Access Policy Manager and Okta complement each other well and provide customers a solution to address identity, access and single sign-on for cloud and on-premises applications regardless of their authentication requirements.

In this integration guide F5 and Okta focus on single sign-on capabilities for on-premises legacy applications that cannot consume a SAML or Claim assertion.  For these legacy applications you can leverage F5’s Access Policy Manger to perform Kerberos Constrained Delegation or Header authentication.

I deviated from the deployment guide and used APM’s per-request policy engine to insert the header versus the iRule.  I prefer this method as it is easier for people new to F5 and it will survive future upgrades.

I’ve provided a video demo below:

Yubikey One-Time-Password Authentication with APM

yubikey_4Well my Yubikey 4 arrived today so I had a chance to play around with their one-time-password capabilities – read about their U2F and APM capabilities here. The primary benefit about OTP over U2F is it’s supported across almost every major browser and OS.  This makes the Yubikey 4 a little more palatable for enterprises – note the Yubikey 4 supports both OTP and U2F.

Jason Rahm posted an article on DevCentral regarding 2FA using Yubikey, YubiCloud and BIG-IP LTM  back in 2013.  I’ve adapted this iRules to use APM Agent Events so we can leverage Yubikeys for 2FA in APM.  For more information on Yubikey OTP clients check out the Getting Started Writing Clients page.


  1. To configure this you’ll need to add the iRule below to your BIG-IP and XXXXXX with your YubiCloud client ID and Secret Key.
  2. Add a data group (yubikey_users) and populate it with username:serial pairs
  3. add an iRule event to your APM VPE
    1. set the name to OTP Valid
    2. set the ID to “otp_verify”
    3. add a branch rule
      1. name it Yes
      2. add an advanced expression of:

expr { [mcget {session.custom.otp_valid} ] == 1}


No too difficult.  Some ways that we could extend this code would be to try multiple cloud instances (api1.yubico.com-api5.yubico.com) and provide a self enrollment page if the user’s serial number is not in our data group – I’m writing an example of this with Google Authenticator and iRules LX so stay tuned.

U2F Authentication with F5 APM and Duo Security

Security-Key-by-Yubico-1000-2016-444x444I’ve been working on Universal 2nd Factor (U2F) authentication today and it’s a very interesting concept.  There is no requirement to enter a 6-digit code for 2nd factor authentication.  The website I’m logging into detects my Yubikey and the key button flashes a blue light.  Press the button and you’re automatically authenticated.

I’ve configured it with some of the websites I use and I’m going to give it a try for the next few weeks to see what I think.  Now, how to make sure I don’t lose it since I don’t normally carry my key chain around…

See the demo below with F5 APM and Duo Security:

iRules LX – AFM/APM Dynamic Firewall Rules

One of the iRules LX projects I’ve been working on is the ability to dynamically open firewall rules based upon a user’s APM Access Session.  In this example the user is external trying to access internal server resources but is blocked by AFM’s default firewall policies.  The user then logs into an APM policy and upon successful login AFM allows that external user access to internal resources.  See the video below for a demo:

iRules do not have direct access to control plane settings.  So prior to TMOS 12.1 this wasn’t easily doable without some serious iRule foo around sideband connections. However, with the introduction of iRules LX we can leverage Node.js to help us interact with the control plane via API calls.

So how does it work?

You still leverate iRules in iRules LX but a new set of commands allow you to pass data from TMM (data plane) to Node.js (control plane) and process that data via javascript – more information in the Getting Started With iRules LX series on DevCentral.

So what does this look like?

Your iRules do not change much except for the addition of the ILX commands:

  • ILX::init – invokes the specified node methods
  • ILX::call – establishes a communication path from an iRules to the node process
  • ILX::notify  – sends a message to the specified node method but does not wait for a response

As an example I’ve included the  iRule used in the AFM/APM dynamic firewall rule project below:

Notice the ILX::init on line 7 – this is configured with the iRules LX plugin name and the iRules LX extension inside our workspace.


What about Node.js

Next, you provide the Node.js methods your iRule calls in the index.js file

Not too hard and it offers a world of possibilities with the inclusion of NPM modules.

Where’s the rest of the code?

For a complete view of the iRules LX code please visit the Github repository.

Prepopulate APM Username From O365 Login

emojiF5’s Access Policy Manager (APM) offers many compelling security and optimization features for Microsoft’s Active Directory Federation Services (ADFS) – more info in the APM/ADFS Deployment Guide.  Odds are if you’re deploying ADFS you’re also using or looking at Office 365.  Normally, when you login to O365 you will be redirect to a Home Realm Discovery page, login.microsoftonline.com.  This page does 2 things:

  1. determines if you’re a federated users (i.e. you use another authentication source like F5, MS ADFS, Okta, …. <insert long list of IDMs on the market today>).
  2. if you’re not a federated user, authenticate you against Azure AD.
Office 365 Empty Login Page

So when you type in your email address Microsoft parses the domain name and automatically redirects you to your local APM/ADFS environment for authentication – pretty nice! However, when you land at the APM login page you notice the username field is empty and you have to enter your username or email address (depending on how APM is configured for authentication) again.

You can fix this by customizing the APM Visual Policy Editor to look for O365 to ADFS logins and prepopulate the username.

Username Prepopulated from O365

So what does the APM configuration look like?

Final VPE

Step 1: Create an object to detect O365 to ADFS Login 

  1. Click the (+) button on the Start fallback branch
  2. In the search box type empty and select the Empty resource then click “Add Item”
  3.  Name the resource O365 Login
  4. Click on the “Branch Rules” tab and click “Add Branch Rule”
  5. Name the branch “Yes” and add the following expression via the advance tab
    • expr {[mcget {session.server.landinguri}] contains “username=”}
  6. Click Save


Detect O365 Login


Step 2: Extract the Username from the O365 Redirect Request

When the user enters their email address into the O365 login this data is included in the GET request back to the APM/ADFS environment.

O365 Login with Federation Redirect
O365 Redirecting to APM/ADFS

To capture this data we’ll need to create perform the following steps:

  1. Click the (+) button on the Yes branch off of O365 Login
  2. In the search box type variable and select the Variable Assign resource then click “Add Item”
  3. Name the resource “set username from O365” and add the following variables

session.logon.last.username = expr {[string range [mcget {session.server.landinguri}] [mcget {session.custom.pos_username}] [mcget {session.custom.pos_at_sign}]] }

session.custom.pos_at_sign = expr {[string first “%40” [mcget {session.server.landinguri}]] – 1}

session.custom.pos_username = expr {[string first “username=” [mcget {session.server.landinguri}]] + 9}

Click “Finished” and then click “Save”

Set Custom Variables

Step 3: Customize the APM Logon Page

Now that we have the username we need to prepopulate it on the APM Logon page and set the form variable to read only.

  1. Click the (+) button on the “set username from O365” fallback branch
  2. Select “Logon Page’ then click “Add Item”
  3. Name the resource “O365 Logon Page”
  4. Change the “Read Only” setting for field 1 to “Yes”
  5. Click  “Save”
APM Logon Page

The remainder of the VPE configurations are basic and are not covered in this article to keep it from being a novel.

That’s it!  Now you’re user’s only have to enter their information once which will definitely make them happier.