Environment settings

Prior to running any code related to desired actions (such as data extraction, data ingestion or thin API extraction) one must establish NXCALS environment by setting up certain properties as follows:

Action/Property	service.url	kafka.producer.bootstrap.servers	spark.servers.url
Data extraction	X
Data ingestion	X	X
Thin API extraction			X

Examples of code snippets for setting these values can be found in our "Quick guide" for Java examples project under this link.

Depending on the environment PRO or TESTBED the following values must be used:

PRO environment

Property	Value
service.url	https://cs-ccr-nxcals5.cern.ch:19093,https://cs-ccr-nxcals5.cern.ch:19094,https://cs-ccr-nxcals6.cern.ch:19093,https://cs-ccr-nxcals6.cern.ch:19094,https://cs-ccr-nxcals7.cern.ch:19093,https://cs-ccr-nxcals7.cern.ch:19094,https://cs-ccr-nxcals8.cern.ch:19093,https://cs-ccr-nxcals8.cern.ch:19094
kafka.producer.bootstrap.servers	cs-ccr-nxcalsstr4.cern.ch:9092,cs-ccr-nxcalsstr5.cern.ch:9092,cs-ccr-nxcalsstr6.cern.ch:9092,cs-ccr-nxcalsstr7.cern.ch:9092
spark.servers.url	cs-ccr-nxcals5.cern.ch:15000,cs-ccr-nxcals6.cern.ch:15000,cs-ccr-nxcals7.cern.ch:15000,cs-ccr-nxcals8.cern.ch:15000

TESTBED environment

Property	Value
service.url	https://cs-ccr-testbed2.cern.ch:19093,https://cs-ccr-testbed2.cern.ch:19094,https://cs-ccr-testbed3.cern.ch:19093,https://cs-ccr-nxcalstbs1.cern.ch:19093,https://cs-ccr-nxcalstbs2.cern.ch:19093
kafka.producer.bootstrap.servers	cs-ccr-nxcalstbs1.cern.ch:9092,cs-ccr-nxcalstbs2.cern.ch:9092,cs-ccr-nxcalstbs3.cern.ch:9092,cs-ccr-nxcalstbs4.cern.ch:9092
spark.servers.url	cs-ccr-nxcalstbs2.cern.ch:15000,cs-ccr-nxcalstbs3.cern.ch:15000,cs-ccr-nxcalstbs4.cern.ch:15000

Please note that in case of Python (using pip install nxcals or Spark bundle) the service.url setting will come from the bundle itself (it is set in spark-defaults.conf). In order to connect to other environments (like TESTBED) you have to set it in the method spark_session_builder.get_or_create(service_url=...).