file_tools – File names, file handlers, et al.

Module containing file manipulation functions

exception pyhetdex.tools.files.file_tools.RegexCompileFail(msg, pattern=None, pos=None)

Error raised when the compilation fails

class pyhetdex.tools.files.file_tools.FileNameRotator(path, keep=-1, touch_files=True, **kwargs)

Given a group of file name templates, creates new file names using a common counter that avoid file name clashes. The file templates follow the standard python string format syntax and expect only one empty. The file name is joined with the path

Parameters:

path : string

path to search for files

keep : int, optional

if not negative, files with the counter less than (max_number - keep) are removed

touch_files : bool, optional

touch the files as soon as their name is created

kwargs : keyword options

template of the files; they are added as attributes to the instance

Raises:

AttributeError

if the name associated to one file is already in use

ValueError

if it’s not possible to format the template

Examples

>>> fnr = FileNameRotator('/path/to/dir', touch_files=False,
...                       log1='file_{}.log', log2='other_{}.log')
>>> print(os.path.basename(fnr.log1))
file_0.log
>>> print(os.path.basename(fnr.log2))
other_0.log

_add_file_names(dfiles)

Add the created file names as instance attributes

Parameters:

dfiles : dictionary

keys and file names

_create_file_names(dfiles, n_files, touch_files)

Create the file names starting the counter from n_files. If any of the created files exist, increase the counter and retry. Once all file names are found, touch them.

Parameters:

dfiles : dictionary

keys and file name templates

n_files : int

starting point for the counter; if any of the files exists, the counter is increased

touch_files : bool

touch the files as soon as their name is created

Returns:

fnames : dictionary

file names with the counter replaced

counter : int

actual counter used

_n_files(files)

Find the number that the next files should have.

Parameters:

files : list

list of file name templates

Returns:

file_counter : int

counter for the new files

_remove_older(files, counter, keep)

All the files with a counter smaller than counter - keep are removed.

Parameters:

files : list

file name templates

counter : int

number used to create the new file

keep : int

number of files to keep beside the newly created one

_validate(dfiles)

Check that none of the keys is already used and that the file templates contain the ‘{}’ string

Parameters:

dfiles : dictionary

keys and file name templates

Raises:

AttributeError

if the name associated to one file is already in use

ValueError

if it’s not possible to format the template

pyhetdex.tools.files.file_tools.prefix_filename(path, prefix)

Split the file name from its path, prepend the prefix to the file name and join it back

Parameters:

path : string

file path and name

prefix : string

string to prepend

Returns:

string

path with the new file name

Examples

>>> print(prefix_filename('/path/to/file.dat', 'new_'))
/path/to/new_file.dat

pyhetdex.tools.files.file_tools.scan_dirs(path, matches='*', exclude=None, recursive=True, followlinks=True, is_matches_regex=False, is_exclude_regex=False)

Generator that searches for and serves directories

Parameters:

path : string

path to search

matches : string or list of strings, optional

Unix shell-style wildcards to filter

exclude : string or list of strings, optional

Unix shell-style wildcards to exclude directories

recursive : bool, optional

search files recursively into path

followlinks : bool, optional

follow symlinks

is_matches_regex, is_exclude_regex : bool, optional

mark the corresponding options as a regex pattern instead of unix shell pattern with possible wildcards

Yields:

dirname : string

name of the directory

pyhetdex.tools.files.file_tools.scan_files(path, matches='*', exclude=None, exclude_dirs=None, recursive=True, followlinks=True, is_matches_regex=False, is_exclude_regex=False, is_exclude_dirs_regex=False)

Generator that search and serves files.

Parameters:

path : string

path to search

matches : string or list of strings, optional

Unix shell-style wildcards to filter; done on the full path+filename

exclude : string or list of strings, optional

Unix shell-style wildcards to exclude files; done on the full path+filename

exclude_dirs : string or list of strings, optional

Unix shell-style wildcards to exclude subdirectories

recursive : bool, optional

search files recursively into path

followlinks : bool, optional

follow symlinks

is_matches_regex, is_exclude_regex, is_exclude_dirs_regex : bool, optional

mark the corresponding options as a regex pattern instead of unix shell pattern with possible wildcards

Yields:

fn : string

name of the file

pyhetdex.tools.files.file_tools.skip_comments(f)

Skip commented lines and returns the file at the start of the first line without comment.

Parameters:

f : file object

Returns:

f : file object

moved to the next non comment line

pyhetdex.tools.files.file_tools.wildcards_to_regex(wildcards, re_compile=True, is_regex=False)

Convert shell wildcard to regex, if is_regex is False

If wildcards is None, a match-nothing regex is used If wildcards is a list, the resulting regex are concatenated with | (or)

Parameters:

wildcards : None, string or list of strings

shell wildcards

re_compile : bool, optional

if true compile the regex before returning

is_regex : bool, optional

interpret the input as a regex

Returns:

regex : string or re.RegexObject

resulting regex

Examples

>>> print(type(wildcards_to_regex("[0-9]*fits")))  
<... '_sre.SRE_Pattern'>
>>> print(wildcards_to_regex("[0-9]*fits",
...       re_compile=False))  
[0-9].*fits\Z(?ms)
>>> print(wildcards_to_regex(None, re_compile=False))
a^
>>> print(wildcards_to_regex(["[0-3]*fits", "[5-9]*fits"],
...       re_compile=False))  
[0-3].*fits\Z(?ms)|[5-9].*fits\Z(?ms)