org.apache.drill.exec.util

Class DrillFileSystemUtil

java.lang.Object

org.apache.drill.exec.util.DrillFileSystemUtil

public class DrillFileSystemUtil
extends Object

In Drill file system all directories and files that start with dot or underscore is ignored. This helper class that delegates all work to list directory and file statuses to FileSystemUtil class, only adding Drill file system filter first.

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.hadoop.fs.PathFilter	DRILL_SYSTEM_FILTER Path filter that skips all files and folders that start with dot or underscore.

Constructor Summary

Constructors

Constructor and Description
DrillFileSystemUtil()

Method Summary

Methods

Modifier and Type	Method and Description
static List<org.apache.hadoop.fs.FileStatus>	listAll(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories and files present in given path applying custom filters if present.
static List<org.apache.hadoop.fs.FileStatus>	listDirectories(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all directories present in given path applying custom filters if present.
static List<org.apache.hadoop.fs.FileStatus>	listFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, boolean recursive, org.apache.hadoop.fs.PathFilter... filters) Returns statuses of all files present in given path applying custom filters if present.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DRILL_SYSTEM_FILTER

public static final org.apache.hadoop.fs.PathFilter DRILL_SYSTEM_FILTER

Path filter that skips all files and folders that start with dot or underscore.

Constructor Detail

DrillFileSystemUtil

public DrillFileSystemUtil()

Method Detail

listDirectories

public static List<org.apache.hadoop.fs.FileStatus> listDirectories(org.apache.hadoop.fs.FileSystem fs,
                                                    org.apache.hadoop.fs.Path path,
                                                    boolean recursive,
                                                    org.apache.hadoop.fs.PathFilter... filters)
                                                             throws IOException

Returns statuses of all directories present in given path applying custom filters if present. Directories that start with dot or underscore are skipped. Will also include nested directories if recursive flag is set to true.

Parameters:

fs - current file system

path - path to directory

recursive - true if nested directories should be included

filters - list of custom filters (optional)

Returns:

list of matching directory statuses

Throws:

IOException

listFiles

public static List<org.apache.hadoop.fs.FileStatus> listFiles(org.apache.hadoop.fs.FileSystem fs,
                                              org.apache.hadoop.fs.Path path,
                                              boolean recursive,
                                              org.apache.hadoop.fs.PathFilter... filters)
                                                       throws IOException

Returns statuses of all files present in given path applying custom filters if present. Files and nested directories that start with dot or underscore are skipped. Will also include files from nested directories if recursive flag is set to true.

Parameters:

fs - current file system

path - path to file or directory

recursive - true if files in nested directories should be included

filters - list of custom filters (optional)

Returns:

list of matching file statuses

Throws:

IOException

listAll

public static List<org.apache.hadoop.fs.FileStatus> listAll(org.apache.hadoop.fs.FileSystem fs,
                                            org.apache.hadoop.fs.Path path,
                                            boolean recursive,
                                            org.apache.hadoop.fs.PathFilter... filters)
                                                     throws IOException

Returns statuses of all directories and files present in given path applying custom filters if present. Directories and files that start with dot or underscore are skipped. Will also include nested directories and their files if recursive flag is set to true.

Parameters:

fs - current file system

path - path to file or directory

recursive - true if nested directories and their files should be included

filters - list of custom filters (optional)

Returns:

list of matching directory and file statuses

Throws:

IOException