This blog is a series of guides which aim to run through the Conditional Access (CA) templates which Microsoft has published under the title Conditional Access templates (Preview). The templates get you up and running with CA but what is the impact of turning on the policies on devices, how can you adjust the templates to work in your environment and what does the end user experience look like on the endpoint?
The series is co-written by Paul Winstanley, a six times Enterprise Mobility MVP based in the UK and Mike Marable, an active member of the MEM community, based in the US.
Conditional Access Overview
Conditional Access rules are a type of if-then statement. If a user wishes to access something then they must complete an action to be able to access. Conditional Access rules get enforced once first-factor authentication has been completed.
Conditional Access use signals to make a decision and then enforce the decision to allow or deny access as per this diagram from the Microsoft documentation.
Microsoft introduced the Conditional Access templates at the end of 2021 as convenient method to deploy policies. These 15 templates cover five different deployment scenarios:
- Secure foundation
- Zero trust
- Remote work
- Protect administrators
- Emerging Threats
This series will cover the 15 templates in detail.
- Part 1 – Block access for unknown or unsupported device platform
- Part 2 – Require multifactor authentication for all users
- Part 3 – Require multifactor authentication for admins
- Part 4 – Require multifactor authentication for guests
- Part 5 – Require multifactor authentication for Azure Management
- Part 6 – Require Password Change for High Risk Users
- Part 7 – Require multifactor authentication for risky sign-ins
Risk
Azure AD Identity Protection includes the ability to identify suspicious actions related to user accounts. Risks can be related to either the user or sign-in. When risks are identified a risk score is attributed and this risk level/state is located in the Risk users or Risky sign-ins reports in the Azure Active Directory\Security blade of the Azure portal.
For both user risk and risky sign-ins, there are two types of detection/calculation. These are Real-time and Offline. Note that your level of reporting on these risks is determined by the Azure Premium licence which you hold as some of these risks are available to Azure AD Premium level 2 licences. There is, also, free risk detection, which is available if you don’t have any Azure AD Premium licencing.
The amount of time a risk is a reported in the console is determined by it’s type. For example, a real-time detection could be in the reports within 5-10 minutes, whilst an offline could take up to 48 hours to appear.
Risky sign-ins
Microsoft has published the sign-in detections in the following article here. They are presented in this blog for completeness.
Premium sign-in risk detections
| Risk detection | Detection type | Description |
|---|---|---|
| Atypical travel | Offline | This risk detection type identifies two sign-ins originating from geographically distant locations, where at least one of the locations may also be atypical for the user, given past behavior. The algorithm takes into account multiple factors including the time between the two sign-ins and the time it would have taken for the user to travel from the first location to the second. This risk may indicate that a different user is using the same credentials. The algorithm ignores obvious “false positives” contributing to the impossible travel conditions, such as VPNs and locations regularly used by other users in the organization. The system has an initial learning period of the earliest of 14 days or 10 logins, during which it learns a new user’s sign-in behavior. |
| Anomalous Token | Offline | This detection indicates that there are abnormal characteristics in the token such as an unusual token lifetime or a token that is played from an unfamiliar location. This detection covers Session Tokens and Refresh Tokens. NOTE: Anomalous token is tuned to incur more noise than other detections at the same risk level. This tradeoff is chosen to increase the likelihood of detecting replayed tokens that may otherwise go unnoticed. Because this is a high noise detection, there’s a higher than normal chance that some of the sessions flagged by this detection are false positives. We recommend investigating the sessions flagged by this detection in the context of other sign-ins from the user. If the location, application, IP address, User Agent, or other characteristics are unexpected for the user, the tenant admin should consider this risk as an indicator of potential token replay. |
| Token Issuer Anomaly | Offline | This risk detection indicates the SAML token issuer for the associated SAML token is potentially compromised. The claims included in the token are unusual or match known attacker patterns. |
| Malware linked IP address | Offline | This risk detection type indicates sign-ins from IP addresses infected with malware that is known to actively communicate with a bot server. This detection matches the IP addresses of the user’s device against IP addresses that were in contact with a bot server while the bot server was active. This detection has been deprecated. Identity Protection will no longer generate new “Malware linked IP address” detections. Customers who currently have “Malware linked IP address” detections in their tenant will still be able to view, remediate, or dismiss them until the 90-day detection retention time is reached. |
| Suspicious browser | Offline | Suspicious browser detection indicates anomalous behavior based on suspicious sign-in activity across multiple tenants from different countries in the same browser. |
| Unfamiliar sign-in properties | Real-time | This risk detection type considers past sign-in history to look for anomalous sign-ins. The system stores information about previous sign-ins, and triggers a risk detection when a sign-in occurs with properties that are unfamiliar to the user. These properties can include IP, ASN, location, device, browser, and tenant IP subnet. Newly created users will be in “learning mode” period where the unfamiliar sign-in properties risk detection will be turned off while our algorithms learn the user’s behavior. The learning mode duration is dynamic and depends on how much time it takes the algorithm to gather enough information about the user’s sign-in patterns. The minimum duration is five days. A user can go back into learning mode after a long period of inactivity. We also run this detection for basic authentication (or legacy protocols). Because these protocols don’t have modern properties such as client ID, there’s limited telemetry to reduce false positives. We recommend our customers to move to modern authentication. Unfamiliar sign-in properties can be detected on both interactive and non-interactive sign-ins. When this detection is detected on non-interactive sign-ins, it deserves increased scrutiny due to the risk of token replay attacks. |
| Malicious IP address | Offline | This detection indicates sign-in from a malicious IP address. An IP address is considered malicious based on high failure rates because of invalid credentials received from the IP address or other IP reputation sources. |
| Suspicious inbox manipulation rules | Offline | This detection is discovered by Microsoft Defender for Cloud Apps. This detection looks at your environment and triggers alerts when suspicious rules that delete or move messages or folders are set on a user’s inbox. This detection may indicate: a user’s account is compromised, messages are being intentionally hidden, and the mailbox is being used to distribute spam or malware in your organization. |
| Password spray | Offline | A password spray attack is where multiple usernames are attacked using common passwords in a unified brute force manner to gain unauthorized access. This risk detection is triggered when a password spray attack has been successfully performed. For example, the attacker is successfully authenticated, in the detected instance. |
| Impossible travel | Offline | This detection is discovered by Microsoft Defender for Cloud Apps. This detection identifies user activities (is a single or multiple sessions) originating from geographically distant locations within a time period shorter than the time it takes to travel from the first location to the second. This risk may indicate that a different user is using the same credentials. |
| New country | Offline | This detection is discovered by Microsoft Defender for Cloud Apps. This detection considers past activity locations to determine new and infrequent locations. The anomaly detection engine stores information about previous locations used by users in the organization. |
| Activity from anonymous IP address | Offline | This detection is discovered by Microsoft Defender for Cloud Apps. This detection identifies that users were active from an IP address that has been identified as an anonymous proxy IP address. |
| Suspicious inbox forwarding | Offline | This detection is discovered by Microsoft Defender for Cloud Apps. This detection looks for suspicious email forwarding rules, for example, if a user created an inbox rule that forwards a copy of all emails to an external address. |
| Mass Access to Sensitive Files | Offline | This detection is discovered by Microsoft Defender for Cloud Apps. This detection looks at your environment and triggers alerts when users access multiple files from Microsoft SharePoint or Microsoft OneDrive. An alert is triggered only if the number of accessed files is uncommon for the user and the files might contain sensitive information |
| Verified threat actor IP | Real-time | This risk detection type indicates sign-in activity that is consistent with known IP addresses associated with nation state actors or cyber crime groups, based on Microsoft Threat Intelligence Center (MSTIC). |
Nonpremium sign-in risk detections
| Risk detection | Detection type | Description |
|---|---|---|
| Additional risk detected | Real-time or Offline | This detection indicates that one of the premium detections was detected. Since the premium detections are visible only to Azure AD Premium P2 customers, they’re titled “additional risk detected” for customers without Azure AD Premium P2 licenses. |
| Anonymous IP address | Real-time | This risk detection type indicates sign-ins from an anonymous IP address (for example, Tor browser or anonymous VPN). These IP addresses are typically used by actors who want to hide their sign-in information (IP address, location, device, and so on) for potentially malicious intent. |
| Admin confirmed user compromised | Offline | This detection indicates an admin has selected ‘Confirm user compromised’ in the Risky users UI or using riskyUsers API. To see which admin has confirmed this user compromised, check the user’s risk history (via UI or API). |
| Azure AD threat intelligence | Offline | This risk detection type indicates user activity that is unusual for the user or consistent with known attack patterns. This detection is based on Microsoft’s internal and external threat intelligence sources. |
Require multifactor authentication for risky sign-ins
This Conditional Access policy requires multifactor authentication to be satisfied when users access Cloud Apps, use User Actions or Authentication context.
It is created in the Azure Portal under the Conditional Access\Policies blade, or in the Microsoft Endpoint Manager console under Devices\Conditional Access.
Click the New policy from template (preview) link.
The policy is available in the Zero trust and Remote Work scenarios, or you can click All to display all the templates.
You can click the View option to view the default settings for the policy.
These are the settings for the Require multifactor authentication for risky sign-ins template.
| Users: | All Users are Included – The current user creating the policy will be excluded |
| Apps: | All apps |
| Sig-in Risk: | Risk levels: High & Medium |
| Access Control: | Grant access – Require multifactor authentication |
When the policy is created it is set as Report-Only. This is good practice, as the impact of enabling the policy can be assessed before doing so.
For our testing, we have created a group called CA Test User Group and added our test users to it. In the policy, under Users, we have chosen Select users and groups and picked the group to filter for. The user creating the policy via the template is excluded by default, but it is recommended to ensure you have a break glass account which can be used to access should the rule cause problems.
In the Access controls\Grant section of the rule, you can see that Require multifactor authentication is selected as the control.
The sign-in risk is defined under the Conditions view and is set to High and Medium for Sign-in risk.
When you are ready to move from the Report-Only mode, switch the toggle to On.
The state of the policy will be reflected in the console
Generating Sign-in risk
To be able to test the rule, you will want to generate risky sign-ins and Microsoft provides guidance on how to do this, ranging from a simple method to a difficult. The methods are listed as:
- Anonymous IP address (easy)
- Unfamiliar sign-in properties (moderate)
- Atypical travel (difficult)
- Leaked credentials in GitHub for workload identities (moderate)
To be able to test it is recommended to use the Anonymous IP address method, as this is very easy to implement. You can take a look at all the sign risk simulations here.
Anonymous IP address
The documentation states the following for anonymous IP address simultion.
‘Completing the following procedure requires you to use:
- The Tor Browser to simulate anonymous IP addresses. You might need to use a virtual machine if your organization restricts using the Tor browser.
- A test account that isn’t yet registered for Azure AD multifactor authentication.
To simulate a sign-in from an anonymous IP, perform the following steps:
- Using the Tor Browser, navigate to https://myapps.microsoft.com.
- Enter the credentials of the account you want to appear in the Sign-ins from anonymous IP addresses report.
The sign-in shows up on the Identity Protection dashboard within 10 – 15 minutes.’
Since our user, Neil Redfearn, is already registered for MFA, the first step to take was to de-register for MFA.
In Azure Active Directory navigate to Users. Locate the user and click on them. Select Authentication Methods and then click Require re-register multifactor authentication.
Once the process has completed, No usable methods will be displayed.
Impact of enabling the policy
For our test user, Neil Redfearn, to be able to report the risky sign-in, we disabled the Require multifactor authentication for all users rule to ensure that a MFA prompt would not occur as this would be the condition that resolved the Require multifactor authentication for risky sign-ins rule and we wanted Neil to be caught by this rule to demo the impact.
After downloading the Tor browser, the first step was to connect.
The browser status will report as Connected.
Following Microsoft’s instructions, we navigated to https://myapps.microsoft.com and were prompted to log in.
We entered Neil’s username and clicked Next.
Followed by the user’s password.
After entering the password, it was reported that Your sign-in was blocked and that unusual activity was detected.
When clicking More details, information on the Error Code (53004), Request id, App id etc can be viewed.
Checking the Sign-in Logs
When it comes to investigating Conditional Access policies, we refer to the Sign-in logs in Azure Active Directory. You can check the sign-ins against an individual user by going into the the Azure Active Directory blade and selecting Users. Search for the user and select, then click the Sign-in logs menu entry.
For the Require multifactor authentication for risky sign-ins, our user Neil gets a Failure. The Failure reason states Cannot configure multi-factor authentication methods due to suspicious activity. Note the Request ID,Correlation ID and Sign-in error code from our More details screenshot earlier.
Under Authentication Details, the Result details reports that the User needs to complete Multi-factor authentication.
Under Conditional Access, the Require multifactor authentication for risky sign-ins rule has a Result of Failure. The Grant Controls state Require multifactor authentication.
Risky Sign-in Details
Since the sign-in was blocked and detected as unusual, we can view the Risky sign-ins logs for further information.
These logs are available under Azure Active Directory\Security\Risky sign-ins.
Neil’s log in has been reported with a Risk State of At risk.
We can click into the sign-in event for more details.
Similar information is reported as in the sign-in event, however we have more detail on the risk. Note the Application, Application ID and IP address information from the More details screenshot previously.
Under Risk info, the Risk level is reported as High, and therefore would be captured by the Require multifactor authentication for risky sign-ins conditional access rule.
Again, under Conditional Access, the Require multifactor authentication for risky sign-ins rule has a Result of Failure. The Grant Controls state Require multifactor authentication.
Re-enable MFA for the user and retest the rule
To ensure that the user is not blocked by the rule and that MFA will allow access when a risky sign-in occurs, we re-registered Neil for multi-factor authentication.
We then returned to the Tor browser and navigated back to https://myapps.microsoft.com and were prompted to log in again.
We entered the username.
Followed by the password for the user.
Since Neil is registered for MFA, he is now challenged.
After completing the MFA request, Neil is able to access My Apps.
Checking the Sign-in logs
For this authentication request, 3 events were recorded in the sign-in logs. We see an Interrupted Status for the sign-in
The Authentication requirement is Multifactor authentication and Additional Details state that This is an expected part of the login flow.
In Authentication Details, we can see that the Correct password has been entered, however Authentication in progress as we have a MFA prompt to fulfil.
Conditional Access reports a Success for the Require multifactor authentication for risky sign-ins rule.
The next two events both report a Success for Status with MFA requirement satisfised by claim in the token for the Additional Details.
The Authentication Details show MFA requirement satisfied.
Again, the Conditional Access reports a Success for the Require multifactor authentication for risky sign-ins rule.
The Risky sign-ins logs show a new At risk sign-in reported.
Under the Conditional Access view, the CA rule reports as Failure, as the user was caught by the rule and was coming from an anonymous IP adress.
However, the user completed MFA as part of the authentication request. Therefore, under Risk info we can see that the Detection Type Anonymous IP address was Remediated. The Risk detail states User passed multifactor authentication.
M365 Defender
The M365 Defender portal shows the user risk in your environment.
Clicking the View all users button redirects us to the Risky users view located in the Azure Portal under Azure Active Directory\Security\Risky users.
Note that the Risky User Details relate to the blocked login via the Tor browser from earlier.
Note the My Apps application in Recent risky sign-ins.
The user Risk level is reported as Medium and therefore would not be captured by the Require Password Change for High Risk Users conditional access rule as this is governed by a condition of high risk and the administrators would not receive an email alert about this medium risk event.
Further reading
Take a look at the following Microsoft documentation if you want to read some more on Risk, Risky sign-ins and Conditional Access.
- What is Conditional Access? – https://learn.microsoft.com/en-us/azure/active-directory/conditional-access/overview
- Common Conditional Access policy: Sign-in risk-based multifactor authentication – https://learn.microsoft.com/en-us/azure/active-directory/conditional-access/howto-conditional-access-policy-risk
- What is risk? – https://learn.microsoft.com/en-us/azure/active-directory/identity-protection/concept-identity-protection-risks
- Simulating risk detections in Identity Protection – https://learn.microsoft.com/en-us/azure/active-directory/identity-protection/howto-identity-protection-simulate-risk
Look out for the next part of the blog.
SCCMentor - Paul Winstanley 
6 comments