fluke.storage
- class fluke.storage.AmazonS3Dir(*args, **kwargs)[source]
This class represents a virtual directory which resides within an Amazon S3 bucket.
- Parameters
auth (AWSAuth) – An
AWSAuth
instance used for authenticating with AWS.bucket (str) – The name of the bucket in which the directory resides.
path (str | None) – The path pointing to the directory. If
None
, then the whole bucket is considered. Defaults toNone
.cache (bool) – Indicates whether it is allowed for any fetched data to be cached for faster subsequent access. Defaults to
False
.create_if_missing (bool) – If set to
True
, then the directory to which the provided path points will be automatically created in case it does not already exist, instead of an exception being thrown. Defaults toFalse
.
- Raises
BucketNotFoundError – The specified bucket does not exist.
InvalidPathError – The provided path does not exist.
InvalidDirectoryError – The provided path does not point to a directory.
- Note
The provided path must not begin with a separator.
Wrong:
/path/to/dir
Right:
path/to/dir
- close() None
Closes all open connections.
- count(recursively: bool = False) int
Returns the total number of files within within the directory.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.- Note
The resulting number may vary depending on the value of parameter
recursively
.
- get_contents(recursively: bool = False, show_abs_path: bool = False) list[str]
Returns a list containing the paths that correspond to the directory’s contents.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Determines whether to include the contents’ absolute path or their path relative to this directory. Defaults to
False
.
- Note
The resulting list may vary depending on the value of parameter
recursively
.
- get_file(path: str) AmazonS3File [source]
Returns the file residing in the specified path as a
AmazonS3File
instance.- Parameters
path (str) – Either the absolute path or the path relative to the directory of the file in question.
- Raises
InvalidPathError – The provided path does not exist.
InvalidFileError – The provided path does not point to a file within the directory.
- Note
The provided path, if absolute, must not begin with a separator.
Wrong:
/path/to/file.txt
Right:
path/to/file.txt
- get_metadata(file_path: str) dict[str, str]
Returns a dictionary containing any metadata associated with the file in question.
- Parameters
file_path (str) – Either the absolute path or the path relative to the directory of the file in question.
- Raises
InvalidFileError – The provided path does not point to a file within the directory.
- get_name() Optional[str]
Returns the name of the directory, or
None
if it’s the root directory.
- get_path() str
Returns the path to the directory.
- get_size(recursively: bool = False) int
Returns the total sum of the sizes of all files within the directory, in bytes.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.- Note
The resulting size may vary depending on the value of parameter
recursively
.
- get_subdir(path: str) AmazonS3Dir [source]
Returns the directory residing in the specified path as an
AmazonS3Dir
instance.- Parameters
path (str) – Either the absolute path or the path relative to the directory of the subdirectory in question.
- Raises
InvalidPathError – The provided path does not exist.
- Note
The provided path, if absolute, must not begin with a separator.
Wrong:
/path/to/dir
Right:
path/to/dir
- is_cacheable() bool
Returns
True
if directory has been defined as cacheable, else returnsFalse
.
- is_file(path: str) bool
Returns
True
if the provided path points to a file, else returnsFalse
.- Parameters
path (str) – Either an absolute path or a path relative to the directory.
- load_metadata(recursively: bool = False) None
Loads any metadata associated with the files within the directory, which can then be accessed via the instance’s
get_metadata
method.- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.- Note
The number of the loaded metadata may vary depending on the value of parameter
recursively
.Any metadata set via the
set_metadata
method will be overridden after invoking this method.
- ls(recursively: bool = False, show_abs_path: bool = False) None
Lists the contents of the directory.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Determines whether to include the contents’ absolute path or their path relative to this directory. Defaults to
False
.
- Note
Empty directories will not be considered if parameter
recursively
is set toTrue
.
- open() None
Opens all necessary connections.
- path_exists(path: str) bool
Returns
True
if the provided path exists within the directory, else returnsFalse
.- Parameters
path (str) – Either an absolute path or a path relative to the directory.
- purge() None
If cacheable, then purges the directory’s cache, else does nothing.
- set_metadata(file_path: str, metadata: dict[str, str]) None
Associates the provided metadata with the file located in the given path.
- Parameters
file_path (str) – Either the absolute path or the path relative to the directory of the file in question.
metadata (dict[str, str]) – A dictionary containing the metadata that are to be associated with the file.
- Raises
NonStringMetadataKeyError – The provided metadata dictionary contains at least one non-string key.
NonStringMetadataValueError – The provided metadata dictionary contains at least one non-string value.
InvalidFileError – The provided path does not point to a file within the directory.
- transfer_to(dst: _Directory, recursively: bool = False, overwrite: bool = False, include_metadata: bool = False, chunk_size: Optional[int] = None, filter: Optional[Callable[[str], bool]] = None, suppress_output: bool = False) bool
Copies all files within this directory into the destination directory. Returns
True
if all files were successfully transferred, else returnsFalse
.- Parameters
dst (_Directory) – A
_Directory
class instance, which represents the transfer operation’s destination.recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.overwrite (bool) – Indicates whether to overwrite any files if they already exist. Defaults to
False
.include_metadata (bool) – Indicates whether any existing metadata are to be assigned to the resulting files. Defaults to
False
.chunk_size (int | None) – If not
None
, then files are transferred in chunks, whose size are equal to this parameter value. Defaults toNone
.filter (Callable[[str], bool] | None) – A filter function to be used in order to exclude certain files from being transferred. This function is considered to receive a single parameter, namely the file’s absolute path within the source directory, and to return a boolean value, based on which it is determined whether the file is to be filtered out during the transfer (
True
) or not (False
). Defaults toNone
.suppress_output (bool) – If set to
True
, then suppresses all output. Defaults toFalse
.
- Raises
InvalidChunkSizeError – Transferring files in chunks of the given size is not supported by the specified destination.
- traverse(recursively: bool = False, show_abs_path: bool = False) Iterator[str]
Returns an iterator capable of traversing the directory by going through the paths of its contents.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Indicates whether it should be displayed the absolute or the relative path of the contents. Defaults to
False
.
- Note
The resulting iterator may vary depending on the value of parameter
recursively
.
- class fluke.storage.AmazonS3File(*args, **kwargs)[source]
This class represents an object which resides within an Amazon S3 bucket.
- Parameters
auth (AWSAuth) – An
AWSAuth
instance used for authenticating with AWS.bucket (str) – The name of the bucket in which the file resides.
path (str) – The path pointing to the file.
load_metadata (bool) – Indicates whether any existing metadata should be loaded during the the file’s instantiation. Defaults to
False
.cache (bool) – Indicates whether it is allowed for any fetched data to be cached for faster subsequent access. Defaults to
False
.
- Raises
BucketNotFoundError – The specified bucket does not exist.
InvalidPathError – The provided path does not exist.
InvalidFileError – The provided path points to a directory.
- Note
The provided path must not begin with a separator.
Wrong:
/path/to/file.txt
Right:
path/to/file.txt
- cat(encoding: str = 'utf-8') None
Prints the contents of a file as text.
- Parameters
encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.- Note
See Standard Encodings for all available encodings.
- close() None
Closes all open connections.
- get_metadata() dict[str, str]
Returns a dictionary containing any metadata associated with the file.
- get_name() str
Returns the file’s name.
- get_path() str
Returns the file’s absolute path.
- get_size() int
Returns the file’s size in bytes.
- is_cacheable() bool
Returns
True
if file has been defined as cacheable, else returnsFalse
.
- load_metadata() None
Loads any metadata associated with the file, which can then be accessed via the instance’s
get_metadata
method.- Note
Any metadata set via the
set_metadata
method will be overridden after invoking this method.
- open() None
Opens all necessary connections.
- purge() None
If cacheable, then purges the file’s cache, else does nothing.
- read() bytes
Reads and returns the file’s contents in bytes.
- read_chunks(chunk_size: int = 8192) Iterator[bytes]
Returns an iterator capable of going through the file’s contents as distinct chunks of bytes.
- Parameters
chunk_size (int) – The file chunk size in bytes. Defaults to
8192
.
- read_lines(chunk_size: Optional[int] = None, encoding: str = 'utf-8') Iterator[str]
Returns an iterator capable of going through the file line-by-line.
- Parameters
chunk_size (int | None) – If not
None
, then the file is read in distinct chunks, whose size are equal to this parameter value. Defaults toNone
.encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.
- Note
See Standard Encodings for all available encodings.
- read_range(start: Optional[int], end: Optional[int]) bytes
Returns the file contents that correspond to the provided byte range.
- Parameters
start (int | None) – The point in file from which to begin reading bytes. If
None
, then begin reading from the start of the file.end (int | None) – The point in file at which to stop reading bytes. If
None
, then stop reading at the end of the file.
- read_text(encoding: str = 'utf-8') str
Reads and returns the file’s contents as text.
- Parameters
encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.- Note
See Standard Encodings for all available encodings.
- set_metadata(metadata: dict[str, str]) None
Associates the provided metadata with this file.
- Parameters
metadata (dict[str, str]) – A dictionary containing the metadata that are to be associated with the file.
- Raises
NonStringMetadataKeyError – The provided metadata dictionary contains at least one non-string key.
NonStringMetadataValueError – The provided metadata dictionary contains at least one non-string value.
- transfer_to(dst: _Directory, overwrite: bool = False, include_metadata: bool = False, chunk_size: Optional[int] = None, suppress_output: bool = False) bool
Copies the file into the provided directory. Returns
True
if everything went as expected, else returnsFalse
.- Parameters
dst (_Directory) – A
_Directory
class instance, which represents the transfer operation’s destination.overwrite (bool) – Indicates whether to overwrite the file if it already exists. Defaults to
False
.include_metadata (bool) – Indicates whether any existing metadata are to be assigned to the resulting file. Defaults to
False
.chunk_size (int | None) – If not
None
, then files are transferred in chunks, whose size are equal to this parameter value. Defaults toNone
.suppress_output (bool) – If set to
True
, then suppresses all output. Defaults toFalse
.
- Raises
InvalidChunkSizeError – Transferring files in chunks of the given size is not supported by the specified destination.
- class fluke.storage.AzureBlobDir(*args, **kwargs)[source]
This class represents a virtual directory which resides within an Azure blob container.
- Parameters
auth (AzureAuth) – An
AzureAuth
instance used for authenticating with Microsoft Azure.container (str) – The name of the container in which the directory resides.
path (str | None) – The path pointing to the directory. If
None
, then the whole container is considered. Defaults toNone
.cache (bool) – Indicates whether it is allowed for any fetched data to be cached for faster subsequent access. Defaults to
False
.create_if_missing (bool) – If set to
True
, then the directory to which the provided path points will be automatically created in case it does not already exist, instead of an exception being thrown. Defaults toFalse
.
- Raises
ContainerNotFoundError – The specified container does not exist.
InvalidPathError – The provided path does not exist.
InvalidDirectoryError – The provided path does not point to a directory.
- Note
The provided path must not begin with a separator.
Wrong:
/path/to/dir
Right:
path/to/dir
- close() None
Closes all open connections.
- count(recursively: bool = False) int
Returns the total number of files within within the directory.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.- Note
The resulting number may vary depending on the value of parameter
recursively
.
- get_contents(recursively: bool = False, show_abs_path: bool = False) list[str]
Returns a list containing the paths that correspond to the directory’s contents.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Determines whether to include the contents’ absolute path or their path relative to this directory. Defaults to
False
.
- Note
The resulting list may vary depending on the value of parameter
recursively
.
- get_file(path: str) AzureBlobFile [source]
Returns the file residing in the specified path as a
AzureBlobFile
instance.- Parameters
path (str) – Either the absolute path or the path relative to the directory of the file in question.
- Raises
InvalidPathError – The provided path does not exist.
InvalidFileError – The provided path does not point to a file within the directory.
- Note
The provided path, if absolute, must not begin with a separator.
Wrong:
/path/to/file.txt
Right:
path/to/file.txt
- get_metadata(file_path: str) dict[str, str]
Returns a dictionary containing any metadata associated with the file in question.
- Parameters
file_path (str) – Either the absolute path or the path relative to the directory of the file in question.
- Raises
InvalidFileError – The provided path does not point to a file within the directory.
- get_name() Optional[str]
Returns the name of the directory, or
None
if it’s the root directory.
- get_path() str
Returns the path to the directory.
- get_size(recursively: bool = False) int
Returns the total sum of the sizes of all files within the directory, in bytes.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.- Note
The resulting size may vary depending on the value of parameter
recursively
.
- get_subdir(path: str) AzureBlobDir [source]
Returns the directory residing in the specified path as an
AzureBlobDir
instance.- Parameters
path (str) – Either the absolute path or the path relative to the directory of the subdirectory in question.
- Raises
InvalidPathError – The provided path does not exist.
- Note
The provided path, if absolute, must not begin with a separator.
Wrong:
/path/to/dir
Right:
path/to/dir
- is_cacheable() bool
Returns
True
if directory has been defined as cacheable, else returnsFalse
.
- is_file(path: str) bool
Returns
True
if the provided path points to a file, else returnsFalse
.- Parameters
path (str) – Either an absolute path or a path relative to the directory.
- load_metadata(recursively: bool = False) None
Loads any metadata associated with the files within the directory, which can then be accessed via the instance’s
get_metadata
method.- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.- Note
The number of the loaded metadata may vary depending on the value of parameter
recursively
.Any metadata set via the
set_metadata
method will be overridden after invoking this method.
- ls(recursively: bool = False, show_abs_path: bool = False) None
Lists the contents of the directory.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Determines whether to include the contents’ absolute path or their path relative to this directory. Defaults to
False
.
- Note
Empty directories will not be considered if parameter
recursively
is set toTrue
.
- open() None
Opens all necessary connections.
- path_exists(path: str) bool
Returns
True
if the provided path exists within the directory, else returnsFalse
.- Parameters
path (str) – Either an absolute path or a path relative to the directory.
- purge() None
If cacheable, then purges the directory’s cache, else does nothing.
- set_metadata(file_path: str, metadata: dict[str, str]) None
Associates the provided metadata with the file located in the given path.
- Parameters
file_path (str) – Either the absolute path or the path relative to the directory of the file in question.
metadata (dict[str, str]) – A dictionary containing the metadata that are to be associated with the file.
- Raises
NonStringMetadataKeyError – The provided metadata dictionary contains at least one non-string key.
NonStringMetadataValueError – The provided metadata dictionary contains at least one non-string value.
InvalidFileError – The provided path does not point to a file within the directory.
- transfer_to(dst: _Directory, recursively: bool = False, overwrite: bool = False, include_metadata: bool = False, chunk_size: Optional[int] = None, filter: Optional[Callable[[str], bool]] = None, suppress_output: bool = False) bool
Copies all files within this directory into the destination directory. Returns
True
if all files were successfully transferred, else returnsFalse
.- Parameters
dst (_Directory) – A
_Directory
class instance, which represents the transfer operation’s destination.recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.overwrite (bool) – Indicates whether to overwrite any files if they already exist. Defaults to
False
.include_metadata (bool) – Indicates whether any existing metadata are to be assigned to the resulting files. Defaults to
False
.chunk_size (int | None) – If not
None
, then files are transferred in chunks, whose size are equal to this parameter value. Defaults toNone
.filter (Callable[[str], bool] | None) – A filter function to be used in order to exclude certain files from being transferred. This function is considered to receive a single parameter, namely the file’s absolute path within the source directory, and to return a boolean value, based on which it is determined whether the file is to be filtered out during the transfer (
True
) or not (False
). Defaults toNone
.suppress_output (bool) – If set to
True
, then suppresses all output. Defaults toFalse
.
- Raises
InvalidChunkSizeError – Transferring files in chunks of the given size is not supported by the specified destination.
- traverse(recursively: bool = False, show_abs_path: bool = False) Iterator[str]
Returns an iterator capable of traversing the directory by going through the paths of its contents.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Indicates whether it should be displayed the absolute or the relative path of the contents. Defaults to
False
.
- Note
The resulting iterator may vary depending on the value of parameter
recursively
.
- class fluke.storage.AzureBlobFile(*args, **kwargs)[source]
This class represents a blob which resides within an Azure blob container.
- Parameters
auth (AzureAuth) – An
AzureAuth
instance used for authenticating with Microsoft Azure.container (str) – The name of the container in which the blob resides.
path (str) – The path pointing to the file.
load_metadata (bool) – Indicates whether any existing metadata should be loaded during the the file’s instantiation. Defaults to
False
.cache (bool) – Indicates whether it is allowed for any fetched data to be cached for faster subsequent access. Defaults to
False
.
- Raises
ContainerNotFoundError – The specified container does not exist.
InvalidPathError – The provided path does not exist.
InvalidFileError – The provided path points to a directory.
- Note
The provided path must not begin with a separator.
Wrong:
/path/to/file.txt
Right:
path/to/file.txt
- cat(encoding: str = 'utf-8') None
Prints the contents of a file as text.
- Parameters
encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.- Note
See Standard Encodings for all available encodings.
- close() None
Closes all open connections.
- get_metadata() dict[str, str]
Returns a dictionary containing any metadata associated with the file.
- get_name() str
Returns the file’s name.
- get_path() str
Returns the file’s absolute path.
- get_size() int
Returns the file’s size in bytes.
- is_cacheable() bool
Returns
True
if file has been defined as cacheable, else returnsFalse
.
- load_metadata() None
Loads any metadata associated with the file, which can then be accessed via the instance’s
get_metadata
method.- Note
Any metadata set via the
set_metadata
method will be overridden after invoking this method.
- open() None
Opens all necessary connections.
- purge() None
If cacheable, then purges the file’s cache, else does nothing.
- read() bytes
Reads and returns the file’s contents in bytes.
- read_chunks(chunk_size: int = 8192) Iterator[bytes]
Returns an iterator capable of going through the file’s contents as distinct chunks of bytes.
- Parameters
chunk_size (int) – The file chunk size in bytes. Defaults to
8192
.
- read_lines(chunk_size: Optional[int] = None, encoding: str = 'utf-8') Iterator[str]
Returns an iterator capable of going through the file line-by-line.
- Parameters
chunk_size (int | None) – If not
None
, then the file is read in distinct chunks, whose size are equal to this parameter value. Defaults toNone
.encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.
- Note
See Standard Encodings for all available encodings.
- read_range(start: Optional[int], end: Optional[int]) bytes
Returns the file contents that correspond to the provided byte range.
- Parameters
start (int | None) – The point in file from which to begin reading bytes. If
None
, then begin reading from the start of the file.end (int | None) – The point in file at which to stop reading bytes. If
None
, then stop reading at the end of the file.
- read_text(encoding: str = 'utf-8') str
Reads and returns the file’s contents as text.
- Parameters
encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.- Note
See Standard Encodings for all available encodings.
- set_metadata(metadata: dict[str, str]) None
Associates the provided metadata with this file.
- Parameters
metadata (dict[str, str]) – A dictionary containing the metadata that are to be associated with the file.
- Raises
NonStringMetadataKeyError – The provided metadata dictionary contains at least one non-string key.
NonStringMetadataValueError – The provided metadata dictionary contains at least one non-string value.
- transfer_to(dst: _Directory, overwrite: bool = False, include_metadata: bool = False, chunk_size: Optional[int] = None, suppress_output: bool = False) bool
Copies the file into the provided directory. Returns
True
if everything went as expected, else returnsFalse
.- Parameters
dst (_Directory) – A
_Directory
class instance, which represents the transfer operation’s destination.overwrite (bool) – Indicates whether to overwrite the file if it already exists. Defaults to
False
.include_metadata (bool) – Indicates whether any existing metadata are to be assigned to the resulting file. Defaults to
False
.chunk_size (int | None) – If not
None
, then files are transferred in chunks, whose size are equal to this parameter value. Defaults toNone
.suppress_output (bool) – If set to
True
, then suppresses all output. Defaults toFalse
.
- Raises
InvalidChunkSizeError – Transferring files in chunks of the given size is not supported by the specified destination.
- class fluke.storage.GCPStorageDir(*args, **kwargs)[source]
This class represents a virtual directory which resides within a Google Cloud Storage bucket.
- Parameters
auth (GCPAuth) – A
GCPAuth
instance used for authenticating with GCP.bucket (str) – The name of the bucket in which the directory resides.
path (str | None) – The path pointing to the directory. If
None
, then the whole bucket is considered. Defaults toNone
.cache (bool) – Indicates whether it is allowed for any fetched data to be cached for faster subsequent access. Defaults to
False
.create_if_missing (bool) – If set to
True
, then the directory to which the provided path points will be automatically created in case it does not already exist, instead of an exception being thrown. Defaults toFalse
.
- Raises
BucketNotFoundError – The specified bucket does not exist.
InvalidPathError – The provided path does not exist.
InvalidDirectoryError – The provided path does not point to a directory.
- Note
The provided path must not begin with a separator.
Wrong:
/path/to/dir
Right:
path/to/dir
- close() None
Closes all open connections.
- count(recursively: bool = False) int
Returns the total number of files within within the directory.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.- Note
The resulting number may vary depending on the value of parameter
recursively
.
- get_contents(recursively: bool = False, show_abs_path: bool = False) list[str]
Returns a list containing the paths that correspond to the directory’s contents.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Determines whether to include the contents’ absolute path or their path relative to this directory. Defaults to
False
.
- Note
The resulting list may vary depending on the value of parameter
recursively
.
- get_file(path: str) GCPStorageFile [source]
Returns the file residing in the specified path as a
GCPStorageFile
instance.- Parameters
path (str) – Either the absolute path or the path relative to the directory of the file in question.
- Raises
InvalidPathError – The provided path does not exist.
InvalidFileError – The provided path does not point to a file within the directory.
- Note
The provided path, if absolute, must not begin with a separator.
Wrong:
/path/to/file.txt
Right:
path/to/file.txt
- get_metadata(file_path: str) dict[str, str]
Returns a dictionary containing any metadata associated with the file in question.
- Parameters
file_path (str) – Either the absolute path or the path relative to the directory of the file in question.
- Raises
InvalidFileError – The provided path does not point to a file within the directory.
- get_name() Optional[str]
Returns the name of the directory, or
None
if it’s the root directory.
- get_path() str
Returns the path to the directory.
- get_size(recursively: bool = False) int
Returns the total sum of the sizes of all files within the directory, in bytes.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.- Note
The resulting size may vary depending on the value of parameter
recursively
.
- get_subdir(path: str) GCPStorageDir [source]
Returns the directory residing in the specified path as an
GCPStorageDir
instance.- Parameters
path (str) – Either the absolute path or the path relative to the directory of the subdirectory in question.
- Raises
InvalidPathError – The provided path does not exist.
- Note
The provided path, if absolute, must not begin with a separator.
Wrong:
/path/to/dir
Right:
path/to/dir
- is_cacheable() bool
Returns
True
if directory has been defined as cacheable, else returnsFalse
.
- is_file(path: str) bool
Returns
True
if the provided path points to a file, else returnsFalse
.- Parameters
path (str) – Either an absolute path or a path relative to the directory.
- load_metadata(recursively: bool = False) None
Loads any metadata associated with the files within the directory, which can then be accessed via the instance’s
get_metadata
method.- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.- Note
The number of the loaded metadata may vary depending on the value of parameter
recursively
.Any metadata set via the
set_metadata
method will be overridden after invoking this method.
- ls(recursively: bool = False, show_abs_path: bool = False) None
Lists the contents of the directory.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Determines whether to include the contents’ absolute path or their path relative to this directory. Defaults to
False
.
- Note
Empty directories will not be considered if parameter
recursively
is set toTrue
.
- open() None
Opens all necessary connections.
- path_exists(path: str) bool
Returns
True
if the provided path exists within the directory, else returnsFalse
.- Parameters
path (str) – Either an absolute path or a path relative to the directory.
- purge() None
If cacheable, then purges the directory’s cache, else does nothing.
- set_metadata(file_path: str, metadata: dict[str, str]) None
Associates the provided metadata with the file located in the given path.
- Parameters
file_path (str) – Either the absolute path or the path relative to the directory of the file in question.
metadata (dict[str, str]) – A dictionary containing the metadata that are to be associated with the file.
- Raises
NonStringMetadataKeyError – The provided metadata dictionary contains at least one non-string key.
NonStringMetadataValueError – The provided metadata dictionary contains at least one non-string value.
InvalidFileError – The provided path does not point to a file within the directory.
- transfer_to(dst: _Directory, recursively: bool = False, overwrite: bool = False, include_metadata: bool = False, chunk_size: Optional[int] = None, filter: Optional[Callable[[str], bool]] = None, suppress_output: bool = False) bool
Copies all files within this directory into the destination directory. Returns
True
if all files were successfully transferred, else returnsFalse
.- Parameters
dst (_Directory) – A
_Directory
class instance, which represents the transfer operation’s destination.recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.overwrite (bool) – Indicates whether to overwrite any files if they already exist. Defaults to
False
.include_metadata (bool) – Indicates whether any existing metadata are to be assigned to the resulting files. Defaults to
False
.chunk_size (int | None) – If not
None
, then files are transferred in chunks, whose size are equal to this parameter value. Defaults toNone
.filter (Callable[[str], bool] | None) – A filter function to be used in order to exclude certain files from being transferred. This function is considered to receive a single parameter, namely the file’s absolute path within the source directory, and to return a boolean value, based on which it is determined whether the file is to be filtered out during the transfer (
True
) or not (False
). Defaults toNone
.suppress_output (bool) – If set to
True
, then suppresses all output. Defaults toFalse
.
- Raises
InvalidChunkSizeError – Transferring files in chunks of the given size is not supported by the specified destination.
- traverse(recursively: bool = False, show_abs_path: bool = False) Iterator[str]
Returns an iterator capable of traversing the directory by going through the paths of its contents.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Indicates whether it should be displayed the absolute or the relative path of the contents. Defaults to
False
.
- Note
The resulting iterator may vary depending on the value of parameter
recursively
.
- class fluke.storage.GCPStorageFile(*args, **kwargs)[source]
This class represents an object which resides within a Google Cloud Storage bucket.
- Parameters
auth (GCPAuth) – A
GCPAuth
instance used for authenticating with GCP.bucket (str) – The name of the bucket in which the file resides.
path (str) – The path pointing to the file.
load_metadata (bool) – Indicates whether any existing metadata should be loaded during the the file’s instantiation. Defaults to
False
.cache (bool) – Indicates whether it is allowed for any fetched data to be cached for faster subsequent access. Defaults to
False
.
- Raises
BucketNotFoundError – The specified bucket does not exist.
InvalidPathError – The provided path does not exist.
InvalidFileError – The provided path points to a directory.
- Note
The provided path must not begin with a separator.
Wrong:
/path/to/file.txt
Right:
path/to/file.txt
- cat(encoding: str = 'utf-8') None
Prints the contents of a file as text.
- Parameters
encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.- Note
See Standard Encodings for all available encodings.
- close() None
Closes all open connections.
- get_metadata() dict[str, str]
Returns a dictionary containing any metadata associated with the file.
- get_name() str
Returns the file’s name.
- get_path() str
Returns the file’s absolute path.
- get_size() int
Returns the file’s size in bytes.
- is_cacheable() bool
Returns
True
if file has been defined as cacheable, else returnsFalse
.
- load_metadata() None
Loads any metadata associated with the file, which can then be accessed via the instance’s
get_metadata
method.- Note
Any metadata set via the
set_metadata
method will be overridden after invoking this method.
- open() None
Opens all necessary connections.
- purge() None
If cacheable, then purges the file’s cache, else does nothing.
- read() bytes
Reads and returns the file’s contents in bytes.
- read_chunks(chunk_size: int = 8192) Iterator[bytes]
Returns an iterator capable of going through the file’s contents as distinct chunks of bytes.
- Parameters
chunk_size (int) – The file chunk size in bytes. Defaults to
8192
.
- read_lines(chunk_size: Optional[int] = None, encoding: str = 'utf-8') Iterator[str]
Returns an iterator capable of going through the file line-by-line.
- Parameters
chunk_size (int | None) – If not
None
, then the file is read in distinct chunks, whose size are equal to this parameter value. Defaults toNone
.encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.
- Note
See Standard Encodings for all available encodings.
- read_range(start: Optional[int], end: Optional[int]) bytes
Returns the file contents that correspond to the provided byte range.
- Parameters
start (int | None) – The point in file from which to begin reading bytes. If
None
, then begin reading from the start of the file.end (int | None) – The point in file at which to stop reading bytes. If
None
, then stop reading at the end of the file.
- read_text(encoding: str = 'utf-8') str
Reads and returns the file’s contents as text.
- Parameters
encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.- Note
See Standard Encodings for all available encodings.
- set_metadata(metadata: dict[str, str]) None
Associates the provided metadata with this file.
- Parameters
metadata (dict[str, str]) – A dictionary containing the metadata that are to be associated with the file.
- Raises
NonStringMetadataKeyError – The provided metadata dictionary contains at least one non-string key.
NonStringMetadataValueError – The provided metadata dictionary contains at least one non-string value.
- transfer_to(dst: _Directory, overwrite: bool = False, include_metadata: bool = False, chunk_size: Optional[int] = None, suppress_output: bool = False) bool
Copies the file into the provided directory. Returns
True
if everything went as expected, else returnsFalse
.- Parameters
dst (_Directory) – A
_Directory
class instance, which represents the transfer operation’s destination.overwrite (bool) – Indicates whether to overwrite the file if it already exists. Defaults to
False
.include_metadata (bool) – Indicates whether any existing metadata are to be assigned to the resulting file. Defaults to
False
.chunk_size (int | None) – If not
None
, then files are transferred in chunks, whose size are equal to this parameter value. Defaults toNone
.suppress_output (bool) – If set to
True
, then suppresses all output. Defaults toFalse
.
- Raises
InvalidChunkSizeError – Transferring files in chunks of the given size is not supported by the specified destination.
- class fluke.storage.LocalDir(*args, **kwargs)[source]
This class represents a directory which resides within the local file system.
- Parameters
path (str) – The path pointing to the directory.
create_if_missing (bool) – If set to
True
, then the directory to which the provided path points will be automatically created in case it does not already exist, instead of an exception being thrown. Defaults toFalse
.
- Raises
InvalidPathError – The provided path does not exist.
InvalidDirectoryError – The provided path does not point to a directory.
- count(recursively: bool = False) int
Returns the total number of files within within the directory.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.- Note
The resulting number may vary depending on the value of parameter
recursively
.
- get_contents(recursively: bool = False, show_abs_path: bool = False) list[str]
Returns a list containing the paths that correspond to the directory’s contents.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Determines whether to include the contents’ absolute path or their path relative to this directory. Defaults to
False
.
- Note
The resulting list may vary depending on the value of parameter
recursively
.
- get_file(path: str) LocalFile [source]
Returns the file residing in the specified path as a
LocalFile
instance.- Parameters
path (str) – Either the absolute path or the path relative to the directory of the file in question.
- Raises
InvalidPathError – The provided path does not exist.
InvalidFileError – The provided path does not point to a file within the directory.
- get_metadata(file_path: str) dict[str, str]
Returns a dictionary containing any metadata associated with the file in question.
- Parameters
file_path (str) – Either the absolute path or the path relative to the directory of the file in question.
- Raises
InvalidFileError – The provided path does not point to a file within the directory.
- get_name() Optional[str]
Returns the name of the directory, or
None
if it’s the root directory.
- get_path() str
Returns the path to the directory.
- get_size(recursively: bool = False) int
Returns the total sum of the sizes of all files within the directory, in bytes.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.- Note
The resulting size may vary depending on the value of parameter
recursively
.
- get_subdir(path: str) LocalDir [source]
Returns the directory residing in the specified path as a
LocalDir
instance.- Parameters
path (str) – Either the absolute path or the path relative to the directory of the subdirectory in question.
- Raises
InvalidPathError – The provided path does not exist.
InvalidDirectoryError – The provided path does not point to a subdirectory within the directory.
- is_file(path: str) bool
Returns
True
if the provided path points to a file, else returnsFalse
.- Parameters
path (str) – Either an absolute path or a path relative to the directory.
- ls(recursively: bool = False, show_abs_path: bool = False) None
Lists the contents of the directory.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Determines whether to include the contents’ absolute path or their path relative to this directory. Defaults to
False
.
- Note
Empty directories will not be considered if parameter
recursively
is set toTrue
.
- path_exists(path: str) bool
Returns
True
if the provided path exists within the directory, else returnsFalse
.- Parameters
path (str) – Either an absolute path or a path relative to the directory.
- set_metadata(file_path: str, metadata: dict[str, str]) None
Associates the provided metadata with the file located in the given path.
- Parameters
file_path (str) – Either the absolute path or the path relative to the directory of the file in question.
metadata (dict[str, str]) – A dictionary containing the metadata that are to be associated with the file.
- Raises
NonStringMetadataKeyError – The provided metadata dictionary contains at least one non-string key.
NonStringMetadataValueError – The provided metadata dictionary contains at least one non-string value.
InvalidFileError – The provided path does not point to a file within the directory.
- transfer_to(dst: _Directory, recursively: bool = False, overwrite: bool = False, include_metadata: bool = False, chunk_size: Optional[int] = None, filter: Optional[Callable[[str], bool]] = None, suppress_output: bool = False) bool
Copies all files within this directory into the destination directory. Returns
True
if all files were successfully transferred, else returnsFalse
.- Parameters
dst (_Directory) – A
_Directory
class instance, which represents the transfer operation’s destination.recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.overwrite (bool) – Indicates whether to overwrite any files if they already exist. Defaults to
False
.include_metadata (bool) – Indicates whether any existing metadata are to be assigned to the resulting files. Defaults to
False
.chunk_size (int | None) – If not
None
, then files are transferred in chunks, whose size are equal to this parameter value. Defaults toNone
.filter (Callable[[str], bool] | None) – A filter function to be used in order to exclude certain files from being transferred. This function is considered to receive a single parameter, namely the file’s absolute path within the source directory, and to return a boolean value, based on which it is determined whether the file is to be filtered out during the transfer (
True
) or not (False
). Defaults toNone
.suppress_output (bool) – If set to
True
, then suppresses all output. Defaults toFalse
.
- Raises
InvalidChunkSizeError – Transferring files in chunks of the given size is not supported by the specified destination.
- traverse(recursively: bool = False, show_abs_path: bool = False) Iterator[str]
Returns an iterator capable of traversing the directory by going through the paths of its contents.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Indicates whether it should be displayed the absolute or the relative path of the contents. Defaults to
False
.
- Note
The resulting iterator may vary depending on the value of parameter
recursively
.
- class fluke.storage.LocalFile(*args, **kwargs)[source]
This class represents a file which resides within the local file system.
- Parameters
path (str) – A path pointing to the file.
- Raises
InvalidPathError – The provided path does not exist.
InvalidFileError – The provided path points to a directory.
- cat(encoding: str = 'utf-8') None
Prints the contents of a file as text.
- Parameters
encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.- Note
See Standard Encodings for all available encodings.
- get_metadata() dict[str, str]
Returns a dictionary containing any metadata associated with the file.
- get_name() str
Returns the file’s name.
- get_path() str
Returns the file’s absolute path.
- get_size() int
Returns the file’s size in bytes.
- read() bytes
Reads and returns the file’s contents in bytes.
- read_chunks(chunk_size: int = 8192) Iterator[bytes]
Returns an iterator capable of going through the file’s contents as distinct chunks of bytes.
- Parameters
chunk_size (int) – The file chunk size in bytes. Defaults to
8192
.
- read_lines(chunk_size: Optional[int] = None, encoding: str = 'utf-8') Iterator[str]
Returns an iterator capable of going through the file line-by-line.
- Parameters
chunk_size (int | None) – If not
None
, then the file is read in distinct chunks, whose size are equal to this parameter value. Defaults toNone
.encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.
- Note
See Standard Encodings for all available encodings.
- read_range(start: Optional[int], end: Optional[int]) bytes
Returns the file contents that correspond to the provided byte range.
- Parameters
start (int | None) – The point in file from which to begin reading bytes. If
None
, then begin reading from the start of the file.end (int | None) – The point in file at which to stop reading bytes. If
None
, then stop reading at the end of the file.
- read_text(encoding: str = 'utf-8') str
Reads and returns the file’s contents as text.
- Parameters
encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.- Note
See Standard Encodings for all available encodings.
- set_metadata(metadata: dict[str, str]) None
Associates the provided metadata with this file.
- Parameters
metadata (dict[str, str]) – A dictionary containing the metadata that are to be associated with the file.
- Raises
NonStringMetadataKeyError – The provided metadata dictionary contains at least one non-string key.
NonStringMetadataValueError – The provided metadata dictionary contains at least one non-string value.
- transfer_to(dst: _Directory, overwrite: bool = False, include_metadata: bool = False, chunk_size: Optional[int] = None, suppress_output: bool = False) bool
Copies the file into the provided directory. Returns
True
if everything went as expected, else returnsFalse
.- Parameters
dst (_Directory) – A
_Directory
class instance, which represents the transfer operation’s destination.overwrite (bool) – Indicates whether to overwrite the file if it already exists. Defaults to
False
.include_metadata (bool) – Indicates whether any existing metadata are to be assigned to the resulting file. Defaults to
False
.chunk_size (int | None) – If not
None
, then files are transferred in chunks, whose size are equal to this parameter value. Defaults toNone
.suppress_output (bool) – If set to
True
, then suppresses all output. Defaults toFalse
.
- Raises
InvalidChunkSizeError – Transferring files in chunks of the given size is not supported by the specified destination.
- class fluke.storage.RemoteDir(*args, **kwargs)[source]
This class represents a directory which resides within a remote file system.
- Parameters
auth (RemoteAuth) – A
RemoteAuth
instance used for authenticating with a remote machine via the SSH protocol.path (str) – The path pointing to the directory.
cache (bool) – Indicates whether it is allowed for any fetched data to be cached for faster subsequent access. Defaults to
False
.create_if_missing (bool) – If set to
True
, then the directory to which the provided path points will be automatically created in case it does not already exist, instead of an exception being thrown. Defaults toFalse
.
- Raises
InvalidPathError – The provided path does not exist.
InvalidDirectoryError – The provided path does not point to a directory.
- close() None
Closes all open connections.
- count(recursively: bool = False) int
Returns the total number of files within within the directory.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.- Note
The resulting number may vary depending on the value of parameter
recursively
.
- get_contents(recursively: bool = False, show_abs_path: bool = False) list[str]
Returns a list containing the paths that correspond to the directory’s contents.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Determines whether to include the contents’ absolute path or their path relative to this directory. Defaults to
False
.
- Note
The resulting list may vary depending on the value of parameter
recursively
.
- get_file(path: str) RemoteFile [source]
Returns the file residing in the specified path as a
RemoteFile
instance.- Parameters
path (str) – Either the absolute path or the path relative to the directory of the file in question.
- Raises
InvalidPathError – The provided path does not exist.
InvalidFileError – The provided path does not point to a file within the directory.
- get_metadata(file_path: str) dict[str, str]
Returns a dictionary containing any metadata associated with the file in question.
- Parameters
file_path (str) – Either the absolute path or the path relative to the directory of the file in question.
- Raises
InvalidFileError – The provided path does not point to a file within the directory.
- get_name() Optional[str]
Returns the name of the directory, or
None
if it’s the root directory.
- get_path() str
Returns the path to the directory.
- get_size(recursively: bool = False) int
Returns the total sum of the sizes of all files within the directory, in bytes.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.- Note
The resulting size may vary depending on the value of parameter
recursively
.
- get_subdir(path: str) RemoteDir [source]
Returns the subdirectory residing in the specified path as a
RemoteDir
instance.- Parameters
path (str) – Either the absolute path or the path relative to the directory of the subdirectory in question.
- Raises
InvalidPathError – The provided path does not exist.
InvalidDirectoryError – The provided path does not point to a subdirectory within the directory.
- is_cacheable() bool
Returns
True
if directory has been defined as cacheable, else returnsFalse
.
- is_file(path: str) bool
Returns
True
if the provided path points to a file, else returnsFalse
.- Parameters
path (str) – Either an absolute path or a path relative to the directory.
- ls(recursively: bool = False, show_abs_path: bool = False) None
Lists the contents of the directory.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Determines whether to include the contents’ absolute path or their path relative to this directory. Defaults to
False
.
- Note
Empty directories will not be considered if parameter
recursively
is set toTrue
.
- open() None
Opens all necessary connections.
- path_exists(path: str) bool
Returns
True
if the provided path exists within the directory, else returnsFalse
.- Parameters
path (str) – Either an absolute path or a path relative to the directory.
- purge() None
If cacheable, then purges the directory’s cache, else does nothing.
- set_metadata(file_path: str, metadata: dict[str, str]) None
Associates the provided metadata with the file located in the given path.
- Parameters
file_path (str) – Either the absolute path or the path relative to the directory of the file in question.
metadata (dict[str, str]) – A dictionary containing the metadata that are to be associated with the file.
- Raises
NonStringMetadataKeyError – The provided metadata dictionary contains at least one non-string key.
NonStringMetadataValueError – The provided metadata dictionary contains at least one non-string value.
InvalidFileError – The provided path does not point to a file within the directory.
- transfer_to(dst: _Directory, recursively: bool = False, overwrite: bool = False, include_metadata: bool = False, chunk_size: Optional[int] = None, filter: Optional[Callable[[str], bool]] = None, suppress_output: bool = False) bool
Copies all files within this directory into the destination directory. Returns
True
if all files were successfully transferred, else returnsFalse
.- Parameters
dst (_Directory) – A
_Directory
class instance, which represents the transfer operation’s destination.recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.overwrite (bool) – Indicates whether to overwrite any files if they already exist. Defaults to
False
.include_metadata (bool) – Indicates whether any existing metadata are to be assigned to the resulting files. Defaults to
False
.chunk_size (int | None) – If not
None
, then files are transferred in chunks, whose size are equal to this parameter value. Defaults toNone
.filter (Callable[[str], bool] | None) – A filter function to be used in order to exclude certain files from being transferred. This function is considered to receive a single parameter, namely the file’s absolute path within the source directory, and to return a boolean value, based on which it is determined whether the file is to be filtered out during the transfer (
True
) or not (False
). Defaults toNone
.suppress_output (bool) – If set to
True
, then suppresses all output. Defaults toFalse
.
- Raises
InvalidChunkSizeError – Transferring files in chunks of the given size is not supported by the specified destination.
- traverse(recursively: bool = False, show_abs_path: bool = False) Iterator[str]
Returns an iterator capable of traversing the directory by going through the paths of its contents.
- Parameters
recursively (bool) – Indicates whether the directory is to be traversed recursively or not. If set to
False
, then only those files that reside directly within the directory are to be considered. If set toTrue
, then all files are considered, no matter whether they reside directly within the directory or within any of its subdirectories. Defaults toFalse
.show_abs_path (bool) – Indicates whether it should be displayed the absolute or the relative path of the contents. Defaults to
False
.
- Note
The resulting iterator may vary depending on the value of parameter
recursively
.
- class fluke.storage.RemoteFile(*args, **kwargs)[source]
This class represents a file which resides within a remote machine’s file system.
- Parameters
auth (RemoteAuth) – A
RemoteAuth
instance used for authenticating with a remote machine via the SSH protocol.path (str) – A path pointing to the file.
cache (bool) – Indicates whether it is allowed for any fetched data to be cached for faster subsequent access. Defaults to
False
.
- Raises
InvalidPathError – The provided path does not exist.
InvalidFileError – The provided path points to a directory.
- cat(encoding: str = 'utf-8') None
Prints the contents of a file as text.
- Parameters
encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.- Note
See Standard Encodings for all available encodings.
- close() None
Closes all open connections.
- get_metadata() dict[str, str]
Returns a dictionary containing any metadata associated with the file.
- get_name() str
Returns the file’s name.
- get_path() str
Returns the file’s absolute path.
- get_size() int
Returns the file’s size in bytes.
- is_cacheable() bool
Returns
True
if file has been defined as cacheable, else returnsFalse
.
- open() None
Opens all necessary connections.
- purge() None
If cacheable, then purges the file’s cache, else does nothing.
- read() bytes
Reads and returns the file’s contents in bytes.
- read_chunks(chunk_size: int = 8192) Iterator[bytes]
Returns an iterator capable of going through the file’s contents as distinct chunks of bytes.
- Parameters
chunk_size (int) – The file chunk size in bytes. Defaults to
8192
.
- read_lines(chunk_size: Optional[int] = None, encoding: str = 'utf-8') Iterator[str]
Returns an iterator capable of going through the file line-by-line.
- Parameters
chunk_size (int | None) – If not
None
, then the file is read in distinct chunks, whose size are equal to this parameter value. Defaults toNone
.encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.
- Note
See Standard Encodings for all available encodings.
- read_range(start: Optional[int], end: Optional[int]) bytes
Returns the file contents that correspond to the provided byte range.
- Parameters
start (int | None) – The point in file from which to begin reading bytes. If
None
, then begin reading from the start of the file.end (int | None) – The point in file at which to stop reading bytes. If
None
, then stop reading at the end of the file.
- read_text(encoding: str = 'utf-8') str
Reads and returns the file’s contents as text.
- Parameters
encoding (str) – The encoding with which to decode the bytes. Defaults to
utf-8
.- Note
See Standard Encodings for all available encodings.
- set_metadata(metadata: dict[str, str]) None
Associates the provided metadata with this file.
- Parameters
metadata (dict[str, str]) – A dictionary containing the metadata that are to be associated with the file.
- Raises
NonStringMetadataKeyError – The provided metadata dictionary contains at least one non-string key.
NonStringMetadataValueError – The provided metadata dictionary contains at least one non-string value.
- transfer_to(dst: _Directory, overwrite: bool = False, include_metadata: bool = False, chunk_size: Optional[int] = None, suppress_output: bool = False) bool
Copies the file into the provided directory. Returns
True
if everything went as expected, else returnsFalse
.- Parameters
dst (_Directory) – A
_Directory
class instance, which represents the transfer operation’s destination.overwrite (bool) – Indicates whether to overwrite the file if it already exists. Defaults to
False
.include_metadata (bool) – Indicates whether any existing metadata are to be assigned to the resulting file. Defaults to
False
.chunk_size (int | None) – If not
None
, then files are transferred in chunks, whose size are equal to this parameter value. Defaults toNone
.suppress_output (bool) – If set to
True
, then suppresses all output. Defaults toFalse
.
- Raises
InvalidChunkSizeError – Transferring files in chunks of the given size is not supported by the specified destination.