Skip to content

Authentication methods

Access to NXCALS system requires either Kerberos or RBAC authentication.

Please note the following rules:

  • For Extraction API in order to use RBAC please set the environment variable NXCALS_RBAC_AUTH=true. This will enable Spark Service that uses RBAC to authenticate against our services in order to generate Hadoop delegation tokens allowing access.
  • For Meta-data API existing RBAC token will take precedence over Kerberos.
  • If RBAC is not present, Kerberos login will be attempted.
  • Please note that Spark currently does not support multi-user authentication from the same session per call. In other words setting a different user per Spark call using existing session has no effect.

Prerequisites:

  • You must request access to NXCALS as described here

RBAC Authentication

Important

RBAC currently works for JAVA APIs only. Work in progress for Python.

In order to use RBAC one has to obtain a valid token. This can be done (for instance) using by location feature or explicit login. For all methods of RBAC login please refer to the RBAC documentation on the wikis. In this snippet you can see how to use explicit RBAC login:

String user = System.getProperty("user.name");
String password = System.getProperty("user.password");

// Enable NXCALS RBAC authentication in Hadoop delegation tokens generation, in Spark (for extraction)
System.setProperty("NXCALS_RBAC_AUTH", "true");

try {
    AuthenticationClient authenticationClient = AuthenticationClient.create();
    RbaToken token = authenticationClient.loginExplicit(user, password);
    ClientTierTokenHolder.setRbaToken(token);
} catch (AuthenticationException e) {
    throw new IllegalArgumentException("Cannot login", e);
}

RBAC token has to be set before the Spark Session is created.

Kerberos Authentication

NXCALS provides seamless Kerberos integration. The system will either use the existing Kerberos token (in the environment) or will try to authenticate the client using provided Kerberos principal & keytab (encrypted file containing user password).

Authenticating with existing Kerberos token (environment shared state)

In essence, all you need to do prior to running your code is to create your Kerberos token:

kinit

We will pick up the Kerberos token and authenticate you automatically. The advantage of this method is the simplicity. In addition, you can setup a cron job to renew the token indefinitely.

Hint

A cron job entry that will re-initialize your krb ticket:

0 8,12,19 * * * kinit -f -r 5d -kt /path/to/your.keytab [username] && aklog
Where you should provide your CERN login as username.

Be aware that the krb ticket has a validity of max 24 hours, thus the provided cron schedule expression should not be less frequent than that.

Authenticating with .keytab file

If you would like to authenticate using your .keytab file, you have to include the following statement in your code:

static {
    System.setProperty("kerberos.principal", "nxcalsuser");            // replace with your username
    System.setProperty("kerberos.keytab", "/opt/nxcalsuser/.keytab");  // replace with the .keytab file location
}

Once this is done, you will have programmatically obtained a Kerberos token. The advantage of this solution is that you rely on your code to obtain the token. It is safer and more reliable.

How to modify the Spark Bundle for Python

The principal and keytab properties can be passed to the Spark context. This can be done by modifying the file nxcals-bundle/conf/spark-defaults.conf in your Python venv:

  • for the NXCALS API services we need to add to spark.driver.extraJavaOptions two additional options:
    -Dkerberos.principal=<user>@CERN.CH -Dkerberos.keytab=/path/to/keytab/my.keytab
    
  • for YARN-specific Kerberos configuration one can use the properties below:
    spark.yarn.keytab /path/to/keytab/my.keytab
    spark.yarn.principal <user>@CERN.CH
    

As well the properties can be passed as command-line options to spark session:

$ pyspark --principal <user>@CERN.CH --keytab /path/to/keytab/my.keytab

Kerberos keytab file generation

The preferred and the easiest method for obtaining keytab file is to generate it on lxplus machine by executing:

cern-get-keytab --user --keytab <keytab.file>

Once the .keytab file is created correctly we can check if obtaining Kerberos ticket works:

kdestroy && kinit -f -r 5d -kt /<path_to_keytab_file>/<user>.keytab <user>
klist

# Correct output

Ticket cache: FILE:/tmp/krb5cc_14420
Default principal: <user>@CERN.CH

Valid starting     Expires            Service principal
07/01/17 14:00:01  07/02/17 15:00:01  krbtgt/CERN.CH@CERN.CH
        renew until 07/06/17 14:00:01

Important

Please note that this file should be well protected as it contains user's password

Troubleshooting Authentication Issues

Kerberos and Hadoop are relatively hard to debug. Messages coming from JVM can be difficult for diagnosing. Extra debugging information can be enabled for the client.

Please add the following settings to your Spark config under /conf/spark-defaults.conf

spark.driver.extraJavaOptions  -Dsun.security.krb5.debug=true -Dsun.security.spnego.debug=true -Djava.security.debug=gssloginconfig,configfile,configparser,logincontext

spark.yarn.appMasterEnv.HADOOP_JAAS_DEBUG true
spark.yarn.am.extraJavaOptions -Dsun.security.krb5.debug=true -Dsun.security.spnego.debug=true -Djava.security.debug=gssloginconfig,configfile,configparser,logincontext
Please also export the debug variable before running spark shell:

export HADOOP_JAAS_DEBUG=true

Kerberos Installation (optional, for non-CERN machines)

1. Install Kerberos client

This will allow access to any Kerberos protected services once a user has successfully logged into the system.

Important

Most likely the Kerberos software installation is not required on standard CERN machines (running Linux, Windows or Mac OSX), and the steps below can be simply omitted.

On the contrary if the installation is necessary please follow instructions below (given for Ubuntu Linux):

Install krb5-user package which provides the basic kinit, klist, kdestroy, and kpasswd clients:

sudo apt-get krb5-user

2. Configure local Kerberos client Configure Kerberos realm and write CERN.CH when requested to complete the realm after executing the following command:

sudo dpkg-reconfigure krb5-config
Full explanation of how to configure Kerberos Linux Client you can find on the following address: https://ubuntu.com/server/docs/service-kerberos-workstation-auth

CERN Grid CA certificate Installation (optional)

Important

This action is considered optional since NXCALS client tries to obtain the CA certificates from the embedded trustStore file that is available on the resources.

If you still need to use your own or existing trustStore that contains the needed CERN certificates, then click on the following panel to see how you can achieve that.

Click to see how to manually install a CERN grid CA certificate...

Warning

Machines with JDK provided by BE-CSS do not require creation of the truststore. It is already preinstalled. The steps below concerning certificate installation can be simply skipped.

1. Download certificate

You can download the 'CERN grid CA certificate' directly to your system from CERN's certificates page (or from direct link).

Hint

Prefer to save the certificate with a name like: CERN_Grid_Certification_Authority.crt, as this name will be used as reference for the rest of this guide.

2. Generate SSL trustStore with JDK's keytool

For this step you need to have Java present on your system. In order to import the CERN grid CA certificate, you need to run the following command via java keytool:

/path/to/jdk/bin/keytool -import -alias cerngridcertificationauthority -file /path/to/CERN_Grid_Certification_Authority.crt -keystore nxcals_cacerts -storepass nxcals -noprompt

3. Reference certificate on NXCALS client applications

Once we have the CERN grid CA certificate imported to the JDK's keystore, we need to reference it on an NXCALS client application startup. In order to do that, we have to export the SSL trustStore and trustStore password as JVM related system properties.

For example, add the following:

System.setProperty("javax.net.ssl.trustStore", "nxcals_cacerts");
System.setProperty("javax.net.ssl.trustStorePassword", "nxcals");