utilities.profiler

cloud.utilities.profiler

Classes

Name	Description
Profiler	A context manager–based profiler to log events and CPU and memory usage

Profiler

cloud.utilities.profiler.Profiler(
    array_uri=None,
    group_uri=None,
    group_member=None,
    id=None,
    period_sec=5,
    trace=False,
)

A context manager–based profiler to log events and CPU and memory usage to a TileDB array.

If the trace parameter is True, CPU and memory usage will be logged to the array every period_sec seconds. This is useful for profiling jobs that are OOM killed.

Examples:

# Basic usage
with Profiler(array_uri="tiledb://array-uri..."):
    # code to profile

# Write custom events
with Profiler(group_uri="tiledb://group-uri...", group_member="log") as prof:
    # code to profile

    # write custom event
    prof.write("my-op", "my-data", "my-extra-data")

    # more code to profile

Methods

Name	Description
write	Write an event to the log array.

write

cloud.utilities.profiler.Profiler.write(op='', data='', extra='')

Write an event to the log array.

When writing large amounts of data, store the data in the extra parameter to improve query performance when the extra data is not needed.

Parameters

Name	Type	Description	Default
op	str	event op, defaults to “”	`''`
data	str	event data, defaults to “”	`''`
extra	str	event extra data, defaults to “”	`''`

Functions

Name	Description
create_log_array	Create an array to hold log events.
write_log_event	Write an event to the log array.

create_log_array

cloud.utilities.profiler.create_log_array(uri)

Create an array to hold log events.

Parameters

Name	Type	Description	Default
uri	str	array URI	required

write_log_event

cloud.utilities.profiler.write_log_event(uri, id, op='', data='', extra='')

Write an event to the log array.

When writing large amounts of data, store the data in the extra parameter to improve query performance when the extra data is not needed.

Parameters

Name	Type	Description	Default
uri	str	array URI	required
id	str	event id	required
op	Optional[str]	event operation, defaults to “”	`''`
data	Optional[str]	event data, defaults to “”	`''`
extra	Optional[str]	event extra data, defaults to “”	`''`