Tuesday, January 17, 2017

Syncing Users and Groups from UNIX into Apace Ranger

The previous blog post showed how to authenticate users logging in to the Apache Ranger admin service via LDAP. An older blog post covered how to sync users and groups from LDAP into Apache Ranger so that they can be used both for authentication and to construct authorization policies. Another option is to sync users and groups from the local UNIX machine into Apache Ranger, something we will cover in this post.

1) Build the Apache Ranger usersync module

Follow the steps in the following tutorial to build Apache Ranger and to setup and start the Apache Ranger Admin service. Once this is done, go back to the Apache Ranger distribution that you have built and copy the usersync module:
  • tar zxvf target/ranger-0.6.0-usersync.tar.gz
  • mv ranger-0.6.0-usersync ${usersync.home}
2) Configure and build the Apache Ranger usersync service 

You will need to install the Apache Ranger Usersync service using "sudo". If the root user does not have a JAVA_HOME property defined, then edit ${usersync.home}/setup.sh + add in, e.g.:
  • export JAVA_HOME=/opt/jdk1.8.0_112
Next edit ${usersync.home}/install.properties and make the following changes:
  • POLICY_MGR_URL = http://localhost:6080
  • SYNC_SOURCE = unix
  • SYNC_INTERVAL = 1 (just for testing purposes....)
Now you can run the setup script via "sudo ./setup.sh". 

3) Start the Usersync service

The Apache Ranger Usersync service can be started via "sudo ./ranger-usersync-services.sh start". After 1 minute (see SYNC_INTERVAL above), it should successfully copy the users/groups from the local UNIX machine into the Apache Ranger Admin. Open a browser and go to "http://localhost:6080", and click on "Settings" and then "Users/Groups". You should see the users and groups synced successfully.
 

Wednesday, December 21, 2016

Authenticating users in the Apache Ranger Admin Service via LDAP

In the summer, I wrote a couple of blog posts covering how to configure and install the Apache Ranger Admin Service and then how to use the Apache Ranger usersync Service to import users and groups from LDAP into the Apache Ranger Admin Service. The Usersync service periodically imports all users and groups that matches the configured search base into the Apache Ranger Admin service. However, this bulk import of users and groups might be unnecessary for your particular requirements. In this post we will show instead how to authenticate users logging in to the Apache Ranger Admin Service UI using LDAP.

1) The OpenDS backend

As with the tutorial on the Apache Ranger usersync service, we will use OpenDS as the LDAP server. It contains a domain called "dc=example,dc=com", and 5 users (alice/bob/dave/oscar/victor) and 2 groups (employee/manager). Victor, Oscar and Bob are employees, Alice and Dave are managers. Here is a screenshot using Apache Directory Studio:


2) Configuring the Apache Ranger Admin Service to use LDAP for authentication

Follow the steps in the following tutorial to build Apache Ranger and to setup and install the Apache Ranger Admin service. Edit 'conf/ranger-admin-site.xml' and change/specify the following configuration values:
  • ranger.authentication.method: LDAP
  • ranger.ldap.url: ldap://localhost:2389
  • ranger.ldap.user.dnpattern: cn={0},ou=users,dc=example,dc=com
  • ranger.ldap.group.searchbase: ou=groups,dc=example,dc=com
  • ranger.ldap.group.searchfilter: (member=cn={1},ou=users,dc=example,dc=com)
  • ranger.ldap.group.roleattribute: cn
  • ranger.ldap.base.dn: dc=example,dc=com
  • ranger.ldap.bind.dn: cn=Directory Manager,dc=example,dc=com
  • ranger.ldap.bind.password: test
Note that the "group" configuration attributes must be specified, even though the group information is not actually used. I've submitted a patch for this which should be fixed for the next Ranger release. Now simply save the changes to 'conf/ranger-admin-site.xml' and start the Apache Ranger Admin service. You should be able to log on to http://localhost:6080/login.jsp using the LDAP credentials store in OpenDS.

Friday, November 11, 2016

Home Realm Discovery in the Apache CXF Fediz IdP

When a client application (secured via either WS-Federation or SAML SSO) redirects a user to the Apache CXF Fediz IdP, the IdP must figure out what the home realm of the user is. If the home realm of the user corresponds to the realm of the IdP, then the IdP can authenticate the user. However, if the home realm does not match that of the IdP, then the IdP has the option to forward the authentication request to a third party IdP for authentication, if it is configured to do this. In this post, we will look at the different options available in the IdP to figure out what the home realm of the user is.

1) The 'whr' query parameter

When using the WS-Federation protocol, the application can specify the home realm of the user by adding the 'whr' query parameter to the URI that the browser is redirected to. Alternatively, the 'whr' query parameter could be added by a reverse proxy sitting in front of the IdP. Here is an example of such a URI including a 'whr' query parameter:
  • https://localhost:45753/fediz-idp-realmb/federation?wa=wsignin1.0&wtrealm=urn%3Aorg%3Aapache%3Acxf%3Afediz%3Aidp%3Arealm-A&wreply=https%3A%2F%2Flocalhost%3A43618%2Ffediz-idp%2Ffederation&whr=urn:org:apache:cxf:fediz:idp:realm-B&wfresh=10&wctx=c07a5b9a-e270-4855-9201-fc1801851cc9
2) The 'hrds' configuration option in the IdP

If no 'whr' query parameter is available (this will always be the case for SAML SSO), then the IdP attempts to find out the home realm of the user by querying the "hrds" property of the IdP. This is a Spring Expression Language expression that is evaluated on the Spring WebFlow RequestContext.

For an example of how this can be used, let's look at the tests in Fediz for the SAML SSO IdP when redirecting to a trusted third party IdP. As there is no 'whr' query parameter for SAML SSO, instead we will define a class with a static method that maps application realms to home realms. The application realm is available in the IdP, as the SAML SSO AuthnRequest is already parsed at this point (it corresponds to the "Issuer" of the AuthnRequest). So we can specify the hrds configuration options in the IdP as follows:
  • <property name="hrds" value="T(org.apache.cxf.fediz.integrationtests.RealmMapper).realms()                                   .get(getFlowScope().get('saml_authn_request').issuer)" />
3) Via a form

If no 'whr' query parameter is available, and no 'hrds' configuration option is specified, then the IdP will display a form where the user can select the home realm. The IdP only does this if the configuration option "provideIdpList" is set to true. If it is set to false, then the current IdP is assumed to be the home realm IdP, unless the configuration option "useCurrentIdp" is also set to "false", in which case an error is displayed. The user can select the home realm in the form corresponding to the known trusted IdP realms of this IdP:


Friday, November 4, 2016

Support for IdP-initiated SAML SSO in Apache CXF

Previous blog posts have covered how to secure your JAX-RS web applications in Apache CXF using SAML SSO. Since the 3.1.8 release, Apache CXF also supports IdP-initiated SAML SSO. The typical use-case for SAML SSO involves the browser invoking on a JAX-RS application, and then being redirected to an IdP for authentication, which subsequently redirects the browser back to the application. However, sometimes a user will log on first to the IdP and then want to invoke on a web application. In this post we will show how to configure SAML SSO for a CXF-based web application to support the IdP-initiated flow, by demonstrating an interop test-case with Okta.

1) Configuring a SAML application in Okta

The first step is to create an account at Okta and configure a SAML application. This process is mapped out at the following link. Follow the steps listed on this page with the following additional changes:
  • Specify the following for the Single Sign On URL and audience URI: http://localhost:8080/fedizdoubleit/racs/sso
  • Specify the following for the default RelayState: http://localhost:8080/fedizdoubleit/app1/services/25
  • Add an additional attribute with name "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/role" and value "Manager".
The RequestAssertionConsumerService will process the SAML Response from Okta. However, it doesn't know where to subsequently send the browser. Therefore, we are configuring the RelayState parameter to encode the URL of the end application. In addition, our test application requires that the user has a specific role to invoke upon it, hence we add a "Manager" attribute with the URI corresponding to a role.

When the application is configured, you will see an option to "View Setup Instructions". Open this link in a new tab and set it aside for the moment - it contains information required when setting up the web application. Now click on the "People" tab and assign the application to the username that you have created at Okta.

2) Setting up the SAML SSO-enabled web application

We will use a trivial "double it" web application which I wrote previously to demonstrate the SAML SSO capabilities of Apache CXF Fediz. The web application is available here. Build the web application and deploy it in Apache Tomcat. You will need to edit 'webapps/fedizdoubleit/WEB-INF/cxf-service.xml'.

a) SamlRedirectBindingFilter configuration changes

First let's look at the changes which are required to the 'SamlRedirectBindingFilter':
  • Remove "idpServiceAddress" and "assertionConsumerServiceAddress". These aren't required as we are only supporting the IdP-initiated flow.
  • Also remove the "signRequest", "signaturePropertiesFile", "callbackHandler", "signatureUsername" and "issuerId" properties.
  • Add <property name="addWebAppContext" value="false"/>
  • Add <property name="supportUnsolicited" value="true"/>

b) RequestAssertionConsumerService (RACS) configuration changes

Now add the following properties to the "RequestAssertionConsumerService":
  • <property name="supportUnsolicited" value="true"/>
  • <property name="idpServiceAddress" value="..."/>
  • <property name="issuerId" value="http://localhost:8080/fedizdoubleit/racs/sso"/>
  • <property name="parseApplicationURLFromRelayState" value="true"/>
Paste in the value for "idpServiceAddress" from the "Identity Provider Single Sign-On URL" given in the "View Setup Instructions" page referenced above.
c) Adding Okta cert into the RACS truststore

As things stand, the SAML Response from Okta to the RequestAssertionConsumerService will fail, as the RACS will not trust the certificate Okta uses to sign the SAML Response. Therefore we need to insert the Okta cert into the truststore of the RACS. Copy the "X.509 Certificate" value from the "View Setup Instructions" page referenced earlier. Create a file called 'webapps/fedizdoubleit/WEB-INF/classes/okta.cert' and paste the certificate contents into this file. Import it into the truststore via:
  • keytool -keystore stsrealm_a.jks -storepass storepass -importcert -file okta.cert
At this point we should be all done. Click on the box for the application you have created in Okta. You should be redirected to the CXF RACS, which validates the SAML Response, and in turn redirects to the application.


Wednesday, November 2, 2016

Client Credentials grant support in the Apache CXF Fediz OIDC service

Apache CXF Fediz ships with a powerful and flexible OpenId Connect (OIDC) service that can be used to implement SSO for your organisation. All of the core OIDC flows are supported - Authorization Code flow, Implicit and Hybrid flows. As OIDC is just an identity layer over OAuth 2.0, it's possible to use Fediz as a purely OAuth 2.0 service as well, and all of the authorization grants defined in the spec are also fully supported. In this post we will look at support for one of these authorization grants in Fediz 1.3.1 - the client credentials grant.

1) The OAuth 2.0 client credentials grant

The client credentials grant is used for when the client is requesting access for a resource that is owned or controlled by that client. There is no enduser in this scenario, unlike say the authorization code flow or implicit flow. The client simply calls the token endpoint of the authorization service using "client_credentials" for the "grant_type" parameter. In addition, the client must authenticate (e.g. by supplying client_id and client_secret parameters). The authorization service authenticates the client and then returns an access token.

2) Supporting the client credentials grant in Fediz OIDC

It's easy to support the client credentials grant in the Fediz OIDC service.

a) Add the ClientCredentialsGrantHandler

Firstly, the ClientCredentialsGrantHandler must be added to the list of grant handlers supported by the token service as follows:

b) Add a way of authenticating the client

The next step is to add a way of authenticating the client credentials. Fediz uses JAAS to make it easy for the deployer to plug in different JAAS LoginModules if required. To configure JAAS, you must specify the name of the JAAS LoginModule in the configuration of the OAuthDataProviderImpl:

c) Example JAAS configuration

For the normal OIDC flows, the Fediz OIDC uses a WS-Federation filter to redirect the browser to the Fediz IdP, where the end user is then ultimately authenticated by the STS that bundles with Fediz. Therefore it seems like a natural fit to re-use the STS to authenticate the client in the Fediz OIDC. Follow steps (a) and (b) above. Start the Fediz STS, but before starting the OIDC service, specify the "java.security.auth.login.config" system property to point to the following JAAS configuration file:

You must substitute the correct port for "${idp.https.port}". The STSLoginModule takes the given username and password supplied by the client and uses them to authenticate to the STS.

Wednesday, October 26, 2016

Switching authentication mechanisms in the Apache CXF Fediz STS

Apache CXF Fediz ships with an Identity Provider (IdP) that can authenticate users via either the WS-Federation or SAML SSO protocols. The IdP delegates user authentication to a Security Token Service (STS) web application using the WS-Trust protocol. The STS implementation in Fediz ships with some sample user data for use in the tests. For a real-world scenario, deployers will have to swap the sample data out for an identity backend (such as Active Directory or LDAP). This post will explain how this can be done, with a particular focus on some recent changes to the STS web application in Fediz to make the process easier.

1) The default STS that ships with Fediz

First let's explain a bit about how the STS is configured by default in Fediz to cater for the testcases.

a) Endpoints and user authentication

The STS must define two distinct set of endpoints to work with the IdP. Firstly, the STS must be able to authenticate the user credentials that are presented to the IdP. Typically this is a Username + Password combination. However, X.509 client certificates and Kerberos tokens are also supported. Note that by default, the STS authenticates usernames and passwords via a simple file local to the STS.

After successful user authentication, a SAML token is returned to the IdP. The IdP then gets another SAML token "on behalf of" the authenticated user for a given realm, authenticating using its own credentials. So we need a second endpoint in the STS to issue this token. By default, the STS requires that the IdP authenticate using TLS client authentication. The security policies are defined in the WSDLs available here.

b) Realms

The Fediz IdP and STS support the concept of authenticating users in different realms. By default, the IdP is configured to authenticate users in "Realm A". This corresponds to a specific endpoint address in the STS. The STS also defines user authentication endpoints in "Realm B" for use in test scenarios involving identity federation between two IdPs.

In addition, the STS defines some configuration to map user identities between realms. In other words, how a principal in one realm should map to another realm, and how the claims in one realm map to those in another realm.

2) Changing the STS in Fediz 1.3.2 to use LDAP

From the forthcoming 1.3.2 release onwards, the Fediz STS web application is a bit easier to customize for your specific deployment needs. Let's see how easy it is to switch the STS to use LDAP.

a) Deploy the vanilla IdP and STS to Apache Tomcat

To start with, we will deploy the STS and IdP containing the sample data to Apache Tomcat.
  • Create a new directory: ${catalina.home}/lib/fediz
  • Edit ${catalina.home}/conf/catalina.properties and append ',${catalina.home}/lib/fediz/*.jar' to the 'common.loader' property.
  • Copy ${fediz.home}/plugins/tomcat/lib/* to ${catalina.home}/lib/fediz
  • Copy ${fediz.home}/idp/war/* to ${catalina.home}/webapps
  • Download and copy the hsqldb jar (e.g. hsqldb-2.3.4.jar) to ${catalina.home}/lib 
  • Copy idp-ssl-key.jks and idp-ssl-trust.jks from ${fediz.home}/examples/sampleKeys to ${catalina.home}
  • Edit the TLS Connector in ${catalina.home}/conf/server.xml', e.g.: <Connector port="8443" protocol="org.apache.coyote.http11.Http11Protocol" maxThreads="150" SSLEnabled="true" scheme="https" secure="true" clientAuth="want" sslProtocol="TLS" keystoreFile="idp-ssl-key.jks" keystorePass="tompass" keyPass="tompass" truststoreFile="idp-ssl-trust.jks" truststorePass="ispass" />
Now start Tomcat and then enter the following in a web browser (authenticating with "alice/ecila" in "realm A" - you should be directed to the URL for the default service application (404, as we have not configured it):

https://localhost:8443/fediz-idp/federation?wa=wsignin1.0&wreply=https%3A%2F%2Flocalhost%3A8443%2Ffedizhelloworld%2Fsecure%2Ffedservlet&wtrealm=urn%3Aorg%3Aapache%3Acxf%3Afediz%3Afedizhelloworld

b) Change the STS authentication mechanism to Active Directory

To simulate an Active Directory instance for demonstration purposes, we will modify some LDAP system tests in the Fediz source that use Apache Directory. Check out the Fediz source and build it via "mvn install -DskipTests". Now go into "systests/ldap" and edit the LDAPTest. "@Ignore" the existing test + uncomment the test which just "sleeps". Also change the "@CreateTransport" annotation to start the LDAP port on "12345" instead of a random port.

Next we'll configure the Fediz STS to use this LDAP instance for authentication. Edit 'webapps/fediz-idp-sts/WEB-INF/cxf-transport.xml'. Change "endpoints/file.xml" to "endpoints/ldap.xml". Next edit 'webapps/fediz-idp-sts/WEB-INF/endpoints/ldap.xml" and just change the port from "389" to "12345".

Now we need to configure a JAAS configuration file, which the STS uses to validate the received Username + Password to LDAP. Copy this file to the "conf" directory of Tomcat, substituting "12345" for "portno". Now restart Tomcat, this time specifying the location of the JAAS configuration file, e.g.:
  • export JAVA_OPTS="-Xmx2048M -Djava.security.auth.login.config=/opt/fediz-apache-tomcat-8.0.37/conf/ldap.jaas"
This is all the changes that are required to swap over to use an LDAP instance for authentication.

Wednesday, September 28, 2016

Securing an Apache Kafka broker - part IV

This is the fourth in a series of articles on securing an Apache Kafka broker. The first post looked at how to secure messages and authenticate clients using SSL. The second post built on the first post by showing how to perform authorization using some custom logic. The third post showed how Apache Ranger could be used instead to create and enforce authorization policies for Apache Kafka. In this post we will look at an alternative authorization solution called Apache Sentry.

1) Build the Apache Sentry distribution

First we will build and install the Apache Sentry distribution. Download Apache Sentry (1.7.0 was used for the purposes of this tutorial). Verify that the signature is valid and that the message digests match. Now extract and build the source and copy the distribution to a location where you wish to install it:
  • tar zxvf apache-sentry-1.7.0-src.tar.gz
  • cd apache-sentry-1.7.0-src
  • mvn clean install -DskipTests
  • cp -r sentry-dist/target/apache-sentry-1.7.0-bin ${sentry.home}
Apache Sentry has an authorization plugin for Apache Kafka, amongst other big data projects. In addition it comes with an RPC service which stores authorization privileges in a database. For the purposes of this tutorial we will just configure the authorization privileges in a configuration file locally to the broker. Therefore we don't need to do any further configuration to the distribution at this point.

2) Configure authorization in the broker

Configure Apache Kafka as per the first tutorial. To enable authorization using Apache Sentry we also need to follow these steps. First edit 'config/server.properties' and add:
  • authorizer.class.name=org.apache.sentry.kafka.authorizer.SentryKafkaAuthorizer
  • sentry.kafka.site.url=file:./config/sentry-site.xml
Next copy the jars from the "lib" directory of the Sentry distribution to the Kafka "libs" directory. Then create a new file in the config directory called "sentry-site.xml" with the following content:

This is the configuration file for the Sentry plugin for Kafka. It essentially says that the authorization privileges are stored in a local file, and that the groups for authenticated users should be retrieved from this file. Finally, we need to specify the authorization privileges. Create a new file in the config directory called "sentry.ini" with the following content:

This configuration file contains three separate sections. The "[users]" section maps the authenticated principals to local groups. The "[groups]" section maps the groups to roles, and the "[roles]" section lists the actual privileges. Now we can start the broker as in the first tutorial:
  • bin/kafka-server-start.sh config/server.properties 
3) Test authorization

Now lets test the authorization logic. Start the producer:
  • bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test --producer.config config/producer.properties
Send a few messages to check that the producer is authorized correctly. Now start the consumer:
  • bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning --consumer.config config/consumer.properties --new-consumer
If everything is configured correctly then it should work as in the first tutorial.