Environment settings

Prior to running any code related to desired actions (such as data extraction, data ingestion or thin API extraction) one must establish NXCALS environment by setting up certain properties as follows:

Action/Property service.url kafka.producer.bootstrap.servers spark.servers.url
Data extraction X
Data ingestion X X
Thin API extraction X

Examples of code snippets for setting these values can be found in our "Quick guide" for Java examples project under this link.

Depending on the environment PRO or TESTBED the following values must be used:

Property Value
service.url https://cs-ccr-nxcals5.cern.ch:19093,https://cs-ccr-nxcals5.cern.ch:19094,https://cs-ccr-nxcals6.cern.ch:19093,https://cs-ccr-nxcals6.cern.ch:19094,https://cs-ccr-nxcals7.cern.ch:19093,https://cs-ccr-nxcals7.cern.ch:19094,https://cs-ccr-nxcals8.cern.ch:19093,https://cs-ccr-nxcals8.cern.ch:19094
kafka.producer.bootstrap.servers cs-ccr-nxcalsstr4.cern.ch:9092,cs-ccr-nxcalsstr5.cern.ch:9092,cs-ccr-nxcalsstr6.cern.ch:9092,cs-ccr-nxcalsstr7.cern.ch:9092
spark.servers.url cs-ccr-nxcals5.cern.ch:15000,cs-ccr-nxcals6.cern.ch:15000,cs-ccr-nxcals7.cern.ch:15000,cs-ccr-nxcals8.cern.ch:15000
Property Value
service.url https://cs-ccr-testbed2.cern.ch:19093,https://cs-ccr-testbed2.cern.ch:19094,https://cs-ccr-testbed3.cern.ch:19093,https://cs-ccr-nxcalstbs1.cern.ch:19093,https://cs-ccr-nxcalstbs2.cern.ch:19093
kafka.producer.bootstrap.servers cs-ccr-nxcalstbs1.cern.ch:9092,cs-ccr-nxcalstbs2.cern.ch:9092,cs-ccr-nxcalstbs3.cern.ch:9092,cs-ccr-nxcalstbs4.cern.ch:9092
spark.servers.url cs-ccr-nxcalstbs2.cern.ch:15000,cs-ccr-nxcalstbs3.cern.ch:15000,cs-ccr-nxcalstbs4.cern.ch:15000

Please note that in case of Python (using pip install nxcals or Spark bundle) the service.url setting will come from the bundle itself (it is set in spark-defaults.conf). In order to connect to other environments (like TESTBED) you have to set it in the method spark_session_builder.get_or_create(service_url=...).