Package cern.nxcals.api.utils

Class HdfsUtils

java.lang.Object

cern.nxcals.api.utils.HdfsUtils

public final class HdfsUtils
extends java.lang.Object

Utility class for interacting with HDFS.

Provides methods to list files in a directory and copy files to a local destination.

Method Summary

Modifier and Type	Method	Description
static void	copyToLocal(java.lang.String source, java.lang.String destination)	Copies files from the given source directory to the local destination.
static java.util.List<org.apache.hadoop.fs.Path>	listFiles(java.lang.String srcDir)	Lists all files in the specified HDFS directory.
static void	setFileSystem(org.apache.hadoop.fs.FileSystem fileSystem)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

listFiles

public static java.util.List<org.apache.hadoop.fs.Path> listFiles(java.lang.String srcDir)

Lists all files in the specified HDFS directory.

Only files are returned; directories are ignored. If the directory is empty, an exception is thrown.

Parameters:

srcDir - the HDFS directory path

Returns:

a list of file paths in the specified directory

Throws:

java.lang.IllegalArgumentException - if the directory is empty

java.io.UncheckedIOException - if an error occurs while accessing HDFS

copyToLocal

public static void copyToLocal(java.lang.String source,
                               java.lang.String destination)

Copies files from the given source directory to the local destination.

Only files will be copied; directories are ignored.

Parameters:

source - the source directory containing the files to be copied

destination - the local destination directory where files will be copied

setFileSystem

public static void setFileSystem(org.apache.hadoop.fs.FileSystem fileSystem)